System Design · Core

Caching

Put hot data closer to the CPU: layers, read/write strategies, eviction, stampede, hot keys, and Redis vs Memcached.

1. Why cache

A cache stores a subset of data in a faster medium (usually RAM) so repeated reads avoid the database or a remote service. You trade freshness and operational complexity for latency and QPS.

Without cache: Client → App → DB (every time) With cache: Client → App → Cache (hit) → done → DB (miss) → fill cache

2. Cache layers

Layer	Where	Typical data
Client / browser	User device	Static assets, HTTP cache
CDN edge	PoP near user	Images, JS, video segments
App local	Process memory	Config, tiny hot maps (careful: inconsistency)
Distributed	Redis / Memcached	Sessions, profiles, feed pages
DB buffer pool	Inside DB	Pages / blocks (transparent)

In interviews, draw at least CDN + distributed cache + DB. Mention local in-process cache only if you discuss invalidation across instances.

3. Read/write strategies

Cache-aside (lazy loading)

read(key): v = cache.get(key) if v: return v v = db.read(key) cache.set(key, v, ttl) return v write(key, v): db.write(key, v) cache.delete(key) # or set; delete is safer with races

Most common interview default. App owns the logic. Risk: stampede on miss.

Write-through

Write hits cache and DB together (cache library or proxy). Reads always see warm data for written keys. Higher write latency; good when read path must stay simple.

Write-back (write-behind)

Write to cache; flush to DB asynchronously. Fast writes, risk of data loss on crash unless you have WAL/replication on the cache tier. Rare for user-critical data; more common for counters/metrics buffers.

Read-through

Cache itself loads from DB on miss (app only talks to cache). Similar to cache-aside with the loader living in the cache layer.

Refresh-ahead

Before TTL expires, asynchronously refresh hot keys so users rarely see a cold miss. Needs a predictor of “hot” keys (access counters, known catalogs). Extra write/load traffic; great for predictable popular items.

Strategy	Read path	Write path	When to pick
Cache-aside	App loads on miss	DB then invalidate/update	Default interview answer
Write-through	Usually warm after writes	Cache + DB together	Need simple reads after every write
Write-behind	From cache	Cache first; async DB	Write speed > durability risk OK
Refresh-ahead	Proactive reload before expiry	Same as aside/through	Predictable hot keys, avoid stampede

4. Eviction: LRU, LFU, TTL

Policy	Evicts	Good for
LRU	Least recently used	General working sets with temporal locality
LFU	Least frequently used	Stable popularity (catalog bestsellers)
TTL	Expired keys	Bounding staleness; combine with LRU
Random / noeviction	—	Tiny caches or fail-on-full (Redis noeviction)

Always set a TTL even with LRU so deleted DB rows do not live forever under a rare key. Size the cache for your hot set, not the full dataset.

Data type	TTL guideline	Why
Session / auth token meta	Minutes to session length	Security + logout correctness
User profile / product page	Minutes to hours	Read-heavy; short stale OK
Feed page / ranked list	Seconds to few minutes	Freshness matters more
Config / feature flags	Seconds to minutes	Fast rollout; small objects
CDN static hashed assets	Days / immutable	Content-addressed filenames

5. Cache stampede (thundering herd)

Popular key expires; thousands of requests miss and hammer the DB.

Mitigations: 1) Soft TTL + background refresh (serve stale while one worker reloads) 2) Request coalescing / singleflight (one loader per key) 3) Probabilistic early expiration (XFetch) 4) Lock around miss: only lock holder loads DB

6. Hot keys and large values

Hot key: one key dominates QPS → replicate it to many cache nodes, or add a local L1, or split the logical key.
Big value: multi-MB blobs waste RAM and network → cache IDs/pointers; fetch blob from object store.
Hot shard: consistent hashing alone is not enough if one key is extreme; need explicit replication.

7. Cache penetration

Attackers (or bugs) request keys that never exist in the DB. Every miss falls through → DB overload. Different from stampede (which hits a real hot key after expiry).

Mitigations: 1) Bloom filter in front: "definitely not present" → 404 without DB 2) Cache negative results (null / empty) with short TTL 3) Auth + rate limit abusive key patterns 4) Validate key space (UUIDs, signed ids) before lookup

8. Local vs distributed cache

	Local (in-process)	Distributed (Redis/Memcached)
Latency	Microseconds	Sub-ms to few ms network
Consistency	Per instance; hard to invalidate fleet-wide	Shared view across apps
Capacity	Limited by one process RAM	Cluster RAM; scalable
Use for	Config, tiny hot maps, L1 in front of Redis	Sessions, shared hot objects

9. Consistency pitfalls

Update DB then fail before invalidating cache → stale forever until TTL.
Invalidate then write DB; concurrent reader refills old value → use versioning or short TTL.
Multi-key invariants (account + balance) → do not cache partial aggregates without care.

10. Cache technologies

Tech	Role	Notes
Redis	Distributed cache / data structures	Default interview pick; Cluster, persistence optional
Memcached	Simple distributed blob cache	Very fast; client-side sharding typical
Varnish	HTTP reverse-proxy cache	Edge/origin HTTP caching; VCL rules
CDN (CloudFront, etc.)	Geo edge cache	Static + some dynamic; see CDN notes
Caffeine / Guava	In-process local cache	L1 in front of Redis

Default interview answer for “distributed cache”: Redis, unless the problem is purely “dumb” object cache at massive QPS. Mention Varnish when the cache key is an HTTP URL at the reverse-proxy layer.

11. Performance impact (why bother)

Path	Ballpark latency	Effect
DB primary read	1–10+ ms (disk/lock)	Limits QPS; expensive under load
Redis GET (same AZ)	~0.2–1 ms	10×–100× cheaper than cold DB
Local L1 hit	<0.01 ms	Removes network hop for ultra-hot keys
CDN edge hit	tens of ms RTT to user	Offloads origin entirely for static

12. What to say in a design

“Cache-aside Redis in front of the primary store for read-heavy keys, TTL plus delete-on-write, singleflight against stampede, negative cache or bloom filter against penetration, and CDN for static. Measure hit rate and p99; size RAM for the hot set.”

Quick revision

Layers: browser → CDN → app → Redis → DB buffer.
Strategies: aside / through / behind / refresh-ahead — pick with a table.
TTL by data type; LRU/LFU for capacity.
Stampede ≠ penetration; blooms + null cache for missing keys.
Local L1 vs distributed: latency vs shared invalidation.
Redis / Memcached / Varnish / CDN — name the layer.
Invalidation races are the hard part — name them.