Capacity Estimation

The Final Division

Capacity estimation is where traffic math becomes hardware:

 nodes = peak QPS ÷ (per-node capacity × safe utilization)

 per-node capacity comes from the component type:
   stateless app server  → CPU-bound rps or memory-per-request
   database              → its own engine limits (next lesson)
   cache                 → memory for working set + hit ratio targets

Worked Example: Sizing Each Tier of RideShare

App tier — assume a typical service instance handles ~500 rps at acceptable p99 (measured in load tests; assumed illustrative here), run at 60% utilization:

 peak rider traffic            ≈ 4k rps
 per node effective            ≈ 500 × 0.6 = 300 rps sustained target
 nodes                         = 4,000 ÷ 300 ≈ 14 → run 16 (N+2 headroom)

Location store — 100k writes/sec of tiny ephemeral values:

 Redis-class node: ~100k simple ops/sec → borderline alone
 → shard by driver ID across ~6 primaries (+ replicas) ≈ 12 nodes
 memory: 170k concurrent trips × ~1 KB live key set ≈ well under one node's RAM;
 ops/sec, not bytes, is the binding constraint here

Cache tier — sized by working set, not QPS:

 hot trip records: 170k active trips × 1 KB          ≈ 170 MB
 user sessions + feed fragments: few GB working set
 → 3 × 16 GB Redis nodes with room to spare

Database — connections often bind before QPS:

 16 app nodes × pool of 20 conns = 320 connections
 Postgres comfortable limit ~300–500 → fits now,
 but the formula shows when pooling/proxying becomes mandatory

The Utilization Ceiling

Never divide by full capacity — running components near 100% removes all slack:

 utilization target     consequence
 95%                    any spike queues immediately; latency cliffs
 70%                    absorbs peaks, rolling deploys, node loss
 50–60%                 aggressive HA tiers (payments)
 
 the "missing" capacity IS the reliability budget

Node-loss math doubles as redundancy sizing: if losing one of N nodes must not breach capacity, then N-1 ≥ required nodes — which is why production fleets carry N+1 minimum regardless of QPS math.

Sanity Check the Result

Estimates earn trust through cross-checks:

 claimed: 16 app nodes serve 4k rps
 check:   250 rps/node average — plausible for CRUD+cache hits ✓
 check:   16 nodes × 8 GB heap working sets fit on standard instances ✓
 check:   cost ≈ 16 × $100/mo ≈ $1.6k/mo — proportionate ✓ (illustrative)

An estimate that fails its own sanity checks points at a wrong input — usually per-node capacity or the peak factor.

Interview Framing

Capacity estimation is the moment whiteboards get concrete. Score pattern: one division per tier, utilization stated explicitly, N+1 noted once, then bind results to decisions (“14→16 nodes means LB + stateless tier from day one”). Interviewers rarely audit exact figures; they verify you divide by something defensible and leave headroom deliberately.