Menu

Earn Premium with Referrals

Invite your friends and earn Premium rewards through our referral program.

See how it works and start inviting friends.

Cross-Region Replication
HLD

Cross-Region Replication

The pipes between regions — async/sync modes, lag realities, and topology choices.

The Modes

 ASYNC replication (the default):
   local commit returns IMMEDIATELY; changes shipped after.
   
   [EU write ✓]──lag──►[US copy]
   RPO = replication lag (seconds typically, minutes worst-case)
   
 SYNC replication:
   commit waits for remote acknowledgement:
   
   [EU write]────waits───►[US ack]──►commit visible
   RPO = ZERO. cost: every write pays cross-region RTT (~100ms+)
   + availability coupling: remote slow/down = writes stall.

 semi-sync variants (ack from one nearby replica) split
 the difference. choose per data class — payments sync,
 content async (RPO lesson's table in motion).

Topologies

 STAR (hub):        all regions replicate via/through one hub
                    simple routing; hub = bottleneck+risk

 FULL MESH:         every region ↔ every region
                    lowest lag pairwise; N×N connection sprawl

 RING/CHAIN:        A→B→C→A
                    minimal links; changes traverse hops
                    (2-hop lag for distant pairs)

 [EU]──►[US]  star with US-hub is common de-facto;
 [ ▼    ▼ ]   multi-master databases often manage mesh
 [APAC]       internally — know what YOUR engine does.

The Lag Reality

 steady-state lag ≠ incident lag:

 normal:      seconds
 under load:  replication falls behind (it shares bandwidth/CPU!)
 bulk ops:    big migrations/backfills flood the pipe
 failover:    lag at death-time = YOUR ACTUAL RPO ⚠

 engineering responses:
 □ LAG BUDGETS as SLIs: alert before users notice staleness
 □ LAG-AWARE FEATURES: "syncing… shows updates from 20s ago"
 □ TRAFFIC SHAPING: prioritize critical streams' bandwidth
 □ BACKPRESSURE on bulk producers during peak

Consistency Consequences

 async replication = eventual consistency ACROSS regions:

 user writes post (EU) → views profile via US replica:
   their own post missing?! ← READ-YOUR-WRITES violation

 mitigations (read-after-write lessons, regional flavor):
 - STICKY reads: route THIS user's reads home briefly
   after their write (session-affinity window)
 - VERSION TOKENS: client carries latest-version stamp;
   replica serves stale? waits/routes to freshest ✓
 - per-entity monotonic guarantees documented honestly

Choosing Per Store

DataModeWhy
Ledger/paymentssync (or single-region-writer)zero loss tolerance
User contentasyncbounded loss acceptable
Sessionsregion-local + global storelocality critical
Search indexesasync pipelinefreshness tolerant
Config/featuresasync + fast-push channelconvergence speed matters

Interview Framing

“Three-region active-active design: how does data flow?” scored shape: mode-per-data-class table, topology choice justified (engine-managed mesh vs explicit star), lag-budget monitoring named, read-your-writes mitigation specified (sticky/version-tokens), failover-time-lag-as-RPO caveat. Replication questions are consistency questions wearing network costumes — answer both layers.

My Private Notes

Notes are auto-saved locally to this device.