System Design · Core

Availability Patterns

Keep serving when parts fail: redundancy, health checks, circuit breakers, timeouts/retries, and graceful degradation.

1. Availability is a design choice

Availability ≈ uptime users can successfully use the product. You raise it by removing single points of failure, failing fast, and shedding load before cascading collapse. CAP reminds you that during partitions you pick consistency or availability — say which your product needs.

SPOF examples: one LB, one primary DB, one AZ, one Redis without replica Fix: redundant instances + health routing + multi-AZ

2. Redundancy

Active-active: all replicas take traffic (stateless web, some caches).
Active-passive: standby ready for failover (some DBs, legacy appliances).
N+1 / N+2: capacity for peak plus failure of one/two nodes.
Multi-AZ / multi-region: survive datacenter loss; region failover is harder (data).

3. Health checks and failover

LB / orchestrator probes /healthz fail → remove from pool, restart or replace pass → slow-start back into rotation DB: leader lease expires → elect new leader → fence old writer

Separate liveness (process stuck?) from readiness (safe to receive traffic?). Deep checks that touch a shared DB can mark an entire fleet unhealthy at once — be careful.

4. Timeouts, retries, backoff

Tool	Rule of thumb
Timeouts	Every remote call has one; tighter than the caller’s budget
Retries	Only on idempotent or safely replayable ops
Exponential backoff + jitter	Avoid synchronized retry storms
Retry budget / cap	Stop amplifying outages

Bad: retry forever immediately on every failure Good: timeout 200ms, retry ≤2 with exp backoff + jitter, then fail propagate error; do not retry non-idempotent payments without key

5. Circuit breaker

When a dependency is sick, stop calling it for a cool-down period so it can recover and so your threads are not stuck.

States: CLOSED --many failures--> OPEN --cooldown--> HALF-OPEN ^ | probe success | +------------- success ---+---------------+ OPEN: fail fast locally (fallback or error) HALF-OPEN: allow limited probes

Pair with bulkheads (isolate thread/connection pools per dependency).
Fallback: cached response, default, or “feature disabled.”

6. Load shedding and degradation

Shed: return 503 early when queues are full; protect core paths.
Degrade: skip recommendations, serve stale cache, disable non-critical widgets.
Priority: login and checkout outrank vanity metrics endpoints.

Full experience: personalized feed + ads + notifications badge Degraded: chronological feed from cache, ads off, badge stale User still "in" the product — better than total outage

7. Idempotency and exactly-once illusion

Retries create duplicates. Design write APIs with idempotency keys so availability tactics do not corrupt money or inventory.

8. Interview checklist

No SPOF on the critical path.
Health-based routing + multi-AZ.
Timeouts everywhere; bounded retries with jitter.
Circuit breaker + fallback for flaky deps.
Explicit degradation mode under overload.

9. Bulkheads and pools

Give each dependency its own connection/thread pool so one slow DB cannot exhaust workers needed for a healthy cache path. Combine with queue length limits: reject early instead of unbounded buffering.

pool_db: max 50 conns, timeout 100ms pool_pay: max 20 conns, timeout 500ms pool_rec: max 10 conns, timeout 50ms (optional feature)

10. SLOs and error budgets

Availability patterns support an SLO (e.g. 99.9% successful reads). Error budgets tell you when to freeze risky deploys. Mention this when interviewers ask how you know the system is “available enough.”

Quick revision

Redundancy removes SPOFs; size for N+1 failures.
Liveness vs readiness; careful with deep health checks.
Timeouts mandatory; retries only when safe + backoff/jitter.
Circuit breaker: closed → open → half-open.
Bulkheads isolate failure domains.
Degrade features before dying entirely.
Idempotency makes retries safe.