Cache Stampede

The Failure Shape

 hot key (homepage payload) expires at T:

 t=T   500 concurrent requests → ALL miss simultaneously
       ──► 500 identical queries slam the database
       ──► db latency spikes under redundant load
       ──► timeouts cascade; OTHER queries suffer too

 every expiry of a popular key = a self-inflicted mini-outage.
 worst variant: mass simultaneous expiry after cache flush/restart
 = full thundering herd onto origin.

Defense 1: Request Coalescing (Mutex)

 only ONE request may refill a given key; others WAIT:

 get(key):
   val = cache.get(key)
   if val: return val
   if lock.acquire("fill:"+key, ttl=3s):     ← single winner
       val = db.query(key)
       cache.set(key, val, jittered_ttl)
       lock.release()
   else:
       sleep(50ms); return get(key)          ← waiters re-check

 result: exactly ONE db query per expired key.
 redis SETNX or library singleflight (Go) implement this.

Defense 2: Probabilistic Early Refresh (XFetch)

 instead of locking, serve stale + refresh EARLY with probability:

 each read: if random() < exp(-k × (age/ttl)²): refresh async
 
 effect far from expiry: ~never refresh early
 approaching expiry: probability rises smoothly
 → herd dissolves into a trickle of early refills,
 no locks, no waiting. elegant math, simple code.

Defense 3: Stale-While-Revalidate Semantics

 never let the hot path block on refill:

 on expiry: return STALE copy immediately (if present)
            trigger background refresh

 users see slightly-old data for one refresh cycle instead of
 queueing behind a db stampede. for most data classes that's
 the right trade — staleness beats unavailability.

Defense 4: Expiry Jitter (Prevention)

 stop synchronized expiry from existing:

 ttl = base + random(0, base×0.1)      per-key jitter
 warming: pre-load critical keys BEFORE they're needed

 also: avoid mass flush events (deployments restarting caches)
 during peak traffic; warm progressively on boot.

Choosing Defenses

Situation	Primary defense
Few very-hot keys	Coalescing/mutex
Many medium keys	TTL jitter + stale-while-revalidate
Cache restarts happen	Progressive warming + SWR
Correctness-sensitive keys	Locking + short stale window

Layer them: jitter prevents most herds; SWR absorbs the rest; mutex covers the stubborn top keys.

Detecting It in Production

 signature in metrics:
 - db query rate SPIKES exactly at TTL boundaries
 - correlated latency spike across unrelated endpoints
   (shared db connection pool starves)
 
 alert on db QPS vs cache-miss-rate ratio;
 a healthy cache makes db load boring — spikes mean herds.

Interview Framing

“What happens when the homepage cache entry expires?” is the planted question. Scored arc: name the stampede, quantify (“500 requests × same query”), then defenses IN ORDER — jitter as prevention, SWR as absorption, coalescing as guarantee — with the one-line XFetch mention for extra credit. Candidates who jump to “add a lock” without naming the problem class miss half the points.