Cache Replication

Why Replicate a Cache

 caches hold DERIVED data (rebuildable), so replication here
 serves different goals than database replication:

 ✓ SURVIVAL:      shard master dies → replica promoted;
                  no cold-start stampede on the db
 ✓ MAINTENANCE:   upgrade/reboot replicas, then switchover
 ✗ READ SCALING:  rarely needed — single cache node already
                  serves ~100k simple ops/sec; spread keys
                  across shards instead

The Topology

 per-shard replication (redis style):

 shard-0a (master) ◄── all reads/writes for its keys
    └─async──► shard-0b (replica)     hot standby
 
 fleet = N independent shards × (1 master + 1 replica)

 promotion handled by sentinel/cluster gossip in seconds.
 clients rediscover topology via MOVED redirects or config endpoints.

Async Lag Implications

 replica trails master by milliseconds (more under load):

 failover moment:
 - writes acked by old master but not shipped → LOST from cache
   → consequence: those keys miss once → refill from db
   → SELF-HEALING because it's a cache! loss is benign here.

 contrast with databases: same loss would be data corruption.
 this asymmetry is WHY cache replication can be casual
 where db sync replication is careful.

 exception: locks/sessions living in cache are NOT benign-loss
 — they need stronger machinery or acceptance of re-auth storms.

Read Replicas for Locality?

 geo-scenario: users in EU reading US-master cache data

 option A: read from EU replicas of US master
   + lower latency reads
   − stale reads (async lag + WAN)
   − still WAN-bound for misses/writes anyway

 usually BETTER: regional cache CLUSTERS filled independently:

 [eu users] ─► [eu cluster] ─miss─► [db/eu-replica]
 [us users] ─► [us cluster] ─miss─► [db/primary]

 each region warms its own copies from source-of-truth —
 simpler than replicating cache-to-cache across oceans,
 staleness bounded by TTLs as usual.

When Cache Replication Is Wrong

 ✗ using replicas to scale READS you could shard instead
 ✗ treating promoted replicas as durable (they aren't)
 ✗ replicating session stores async then wondering why
   logged-out users appear after failover (write loss!)
 
 sessions need either sync replication, sticky+drain strategies,
 or acceptance that failover logs everyone out (sometimes OK!
 — decide explicitly).

Operational Notes

 - replicate BEFORE scaling shards: rebalance + replication
   simultaneously is an incident generator
 - monitor REPLICATION LAG per shard; sustained lag = capacity smell
 - test failover quarterly: promote replica under synthetic load,
   watch stampede metrics — drills beat hopes

Interview Framing

“Should we add cache replicas?” scored answer starts with NO-unless-survival (“reads scale by sharding; replicas buy availability”), explains why async write-loss is tolerable for derived data but NOT sessions, and offers regional clusters as the locality answer instead of cross-region replicas. Knowing when NOT to apply a familiar pattern is the tested judgment.