Menu

Earn Premium with Referrals

Invite your friends and earn Premium rewards through our referral program.

See how it works and start inviting friends.

Wide-Column Stores
HLD

Wide-Column Stores

Cassandra/Bigtable — massive write throughput through partitioning-first design.

The Model

 rows with MANY columns, sparsely populated, stored per-partition:

 partition key: driver_id (determines which node owns the row)

 ┌─────────────┬──────────────┬──────────┬─────────┐
 │ driver_id   │ ts           │ lat      │ lng     │   ← clustering:
 │ d-9         │ 10:00:01     │ 51.5     │ -0.09   │     sorted within
 │ d-9         │ 10:00:04     │ 51.5     │ -0.08   │     partition!
 │ d-9         │ 10:00:07     │ ...      │         │
 └─────────────┴──────────────┴──────────┴─────────┘

 PRIMARY KEY ((driver_id), ts)
              ▲ partition    ▲ clustering key = sort order inside

 reads: "driver d-9's pings 10:00–10:05" → ONE partition,
        sequential range scan. designed-in efficiency.

The Design Philosophy: Query First

 Cassandra tables are QUERY-SPECIFIC, not entity-generic:

 // table PER query pattern:
 trips_by_driver   PK((driver_id), created_at)   → driver's history
 trips_by_rider    PK((rider_id), created_at)    → rider's history
 trips_by_city_day PK((city, day), driver_id)    → city dashboards

 the SAME trip written to THREE tables (denormalized by design!)
 because each answers a different question instantly.

 relational asks: model entities, then query any way.
 wide-column asks: list queries FIRST, build a table per pattern.

Why Writes Scale So Well

 the machinery behind ~1M writes/sec clusters:

 - ANY node can accept ANY write (no coordinator bottleneck)
   → writes go to partition owner + replicas via consistent hashing
 - MEMTABLE + SSTable architecture:
     writes append to in-memory memtable + commit log (fast!)
     flushed periodically to immutable SSTables on disk
 - NO random-read-on-write penalty (vs B-tree page updates)
 - LSM compaction merges SSTables in background

 consequence: writes are nearly free; READS pay compaction
 and bloom-filter lookup costs — inverted economics vs B-tree RDBMS.

Consistency: Tunable Per Query

 replication factor RF=3 (each row on 3 nodes):

 WRITE consistency ONE / QUORUM / ALL
 READ consistency  ONE / QUORUM / ALL

 R + W > RF guarantees read-your-writes:
   W=2, R=2, RF=3 → quorum both ways ✓

 production default: QUORUM for critical, ONE for telemetry.
 per-query choice = explicit consistency engineering.

When Wide-Column Wins vs Fails

 ✓ time-series/metrics at massive scale
 ✓ location pings, telemetry, IoT streams
 ✓ messaging histories (per-user partitions)
 ✓ write-heavy event capture

 ✗ ad-hoc queries not designed upfront (no joins, few secondaries)
 ✗ strong multi-entity transactions (limited batch atomicity)
 ✗ small datasets (operational overhead unjustified)
 ✗ hot single partitions (celebrity driver melts one node)

The Hot Partition Caveat

 partition key choice remains THE decision:

 ✗ PK((city), ...) → NYC dwarf every other partition
 ✓ PK((city, day), ...) or bucketed: PK((city, bucket), ...)
 
 monitoring per-partition size/load is mandatory;
 skew mitigation (salting/bucketing) planned BEFORE launch,
 since repartitioning later is painful.

Interview Framing

“Store driver locations at 100k pings/sec” is the canonical wide-column question. Scored shape: partition-key design from queries (“by driver+time window”), table-per-query denormalization acknowledged, LSM write-path explanation, tunable consistency with quorum math, hot-partition caveat volunteered. This family rewards candidates who think access-patterns-first — exactly what it demands in production.

My Private Notes

Notes are auto-saved locally to this device.