Menu

Earn Premium with Referrals

Invite your friends and earn Premium rewards through our referral program.

See how it works and start inviting friends.

Polyglot Persistence
HLD

Polyglot Persistence

Multiple stores, each doing what it does best — orchestrating a database portfolio without losing your mind.

The Premise

 no single store serves all workloads well.
 polyglot persistence = DELIBERATE portfolio:

 ┌─────────────────────────────────────────────────┐
 │ RideShare storage portfolio                     │
 │                                                 │
 │ Postgres      trips, payments, users   (truth)  │
 │ Cassandra     driver locations          (hot)   │
 │ Redis         sessions, cache, locks    (fast)  │
 │ Elasticsearch catalog search           (derived)│
 │ Kafka         events pipeline           (flow)  │
 │ Warehouse     analytics                 (batch) │
 └─────────────────────────────────────────────────┘

 each justified by ITS access patterns and scale numbers —
 never by novelty.

The Architecture That Makes It Workable

 one source of truth; everything else DERIVED:

 [postgres: TRUTH]
     │ CDC (change-data-capture)
     ├──► [elasticsearch]  search index, rebuildable
     ├──► [warehouse]      analytics, rebuildable  
     ├──► [cache invalidation]  redis freshness
     
 [services] ──► [cassandra] locations (own lifecycle, TTL'd)
            ──► [kafka]     domain events

 the golden rule: derived stores must be REBUILDABLE from truth.
 when elasticsearch corrupts → reindex from postgres → done.
 no panic. this property is the whole ballgame.

The Real Costs (the honest section)

CostReality
Operational surfaceN backup strategies, N upgrade cycles, N failure modes
Consistency managementCross-store sync = eventual everywhere
Team skillsHiring/rotating through 5 technologies hurts
Join impossibilityQuestions spanning stores = app-level stitching
 every additional store needs a DEFENSE in design review:
 "what measured workload forces this? what breaks if we
 don't add it?" if answers are vague — don't add it.

 the N-store portfolio is EARNED incrementally:
 start Postgres+Redis. add stores as MEASURED pain demands.

Cross-Store Query Patterns

 "show trip with driver location and search-relevant tags"
 spans three stores. solutions ranked:

 1. DENORMALIZE at write time: trip doc carries location snapshot
    + tags (stale-tolerant display data)
 2. APP-LEVEL FAN-OUT: parallel reads, merge in service
    (latency = slowest branch)
 3. STREAMING PROJECTION: kafka job maintains merged read model
 
 option 1 is usually right for display;
 option 3 for complex composite views (CQRS territory).

Governance That Keeps It Sane

 □ store registry: which store owns which data class, WHY
 □ rebuildability test: quarterly restore-from-truth drill
 □ consistency SLAs per pipeline documented (lag budgets)
 □ deprecation path defined BEFORE adding each store
 □ single team owns each sync pipeline (no orphan pipelines)

 portfolios rot without governance: orphaned indexes,
 nobody-knows-why stores, undocumented sync lag surprises.

Interview Framing

Large designs are scored ON polyglot decomposition: name stores per data class with one-line justifications tied to numbers (“locations: 100k writes/s keyed lookups → Cassandra”), then demonstrate the maturity markers — CDC-derived-read-stores pattern, rebuildability guarantee, and restraint (“start with two stores; add on evidence”). The combination of decomposition AND restraint is the senior signature.

My Private Notes

Notes are auto-saved locally to this device.