Menu

Earn Premium with Referrals

Invite your friends and earn Premium rewards through our referral program.

See how it works and start inviting friends.

Hash Sharding
HLD

Hash Sharding

Uniform distribution through hashing — why it spreads load, and the range-scan price it charges.

The Mechanism

 shard = hash(key) mapped to nodes:

 node = hash(user_id) mod N            (naive)
 or via consistent-hash ring / slot table (production)

 hash scatters SIMILAR keys FAR apart:

 user_1001 → hash → shard 3
 user_1002 → hash → shard 0      ← adjacent ids, distant shards
 user_celebrity → hash → wherever (no special treatment!)

 THE POINT: distribution independent of key semantics.
 insertion patterns can't create hotspots.

What Hash Buys

PropertyWhy
Even loadGood hashes ≈ uniform spread; no range hotspots
No ordering leaksSequential ids don’t cluster writes on one node
Simple routingPure function — no lookup table needed
Write scalingAppends land everywhere; B-trees stay balanced-ish
 contrast with RANGE sharding's failure mode:
 sequential ids + range partitions = ALL inserts hit the
 LAST shard (rightmost range) forever. hash kills this.

What Hash Costs

 ✗ RANGE QUERIES DIE:
   "users 1–10000" or "trips created in March"
   → scatter-gather across ALL shards, then merge.
   
   time-series workloads suffer most:
   range sharding keeps recent data together (fast scans);
   hash shatters it everywhere.

 ✗ ORDERING LOST: pagination by id needs per-shard merges.

 ✗ RESIZE PAIN (modulo variant): N→N+1 remaps ~everything;
   mitigated by consistent hashing/slots — but that's
   infrastructure you must adopt deliberately.

The Hybrid Patterns That Fix Range Needs

 COMPOUND KEYS: hash the ENTITY, keep time as clustering:

   PK((hash(rider_id)), created_at DESC)
    → rider lookups route to ONE shard ✓
    → their trips still scan newest-first WITHIN partition ✓

 TIME-BUCKETED HASH for pure time-series:
   shard = hash(device_id) — device's history co-located,
   queries by device+time-range stay single-shard.
   global time-scans go to warehouse instead. honest boundary.

Choosing Hash vs Range

WorkloadWinner
Point lookups by entity idHash
Uniform write pressure, sequential idsHash
Time-range scans/analyticsRange
Expiring old data by ageRange (drop old partitions)
Mixed: entity-hot + time-coldCompound hybrid

Interview Framing

“Shard by what?” scored answers justify hash with BOTH sides: the spread benefit AND the range-query cost named unprompted, plus the compound-key escape hatch (“hash rider_id as partition key, created_at as sort key — both query shapes survive”). Candidates presenting hash as free wins get probed until they find the range-scan hole; volunteer it and keep the room.

My Private Notes

Notes are auto-saved locally to this device.