Bottleneck Analysis

The Principle

 system throughput = the SLOWEST shared component.
 everything else is idle potential.

 [api 20k rps]──►[cache 500k rps]──►[db 2k rps]──►...
                                      ▲
                            system does 2k. scaling the api
                            or cache changes NOTHING.

Scaling effort spent anywhere but the bottleneck is waste — the most common architecture-review finding.

Finding It: Measure the Chain

 per-request resource profile (traces, not vibes):

 GET /trip/123   p50 breakdown:
   gateway        3 ms     cpu-light
   authn          4 ms
   cache read     1 ms     hit rate 92%
   db query      38 ms  ◄── dominates: THE bottleneck
   render         2 ms

 also check UTILIZATION at target load:
   db cpu 85%, connections 90% of pool  ← saturated
   app cpu 15%                          ← headroom wasted upstream

The Usual Suspects

Bottleneck	Signature	First fixes
Database	High DB CPU, slow queries, connection waits	Indexes, cache, replicas
Connection pools	Requests queue at pool, DB healthy	Bigger pool, fewer round trips
Locks/contention	Latency spikes, low CPU everywhere	Shrink critical sections
Network egress	NIC saturation, bandwidth bills	Compression, CDN, pagination
Single-threaded tier	One process pegged, siblings idle	Shard it or go multi-process
Downstream API	Timeouts correlate with vendor	Cache, bulkhead, fallback

Amdahl’s Law: The Ceiling on Any Fix

 if fraction f of time is parallelizable/improvable,
 max speedup from improving it:

 speedup ≤ 1 / ((1-f) + f/s)

 db is 80% of latency (f=0.8), shard it 10x better:
   speedup = 1 / (0.2 + 0.08) ≈ 3.6x    NOT 10x
 
 and once fixed, the NEXT bottleneck (the remaining 20%)
 becomes the wall. bottlenecks MIGRATE; analysis is continuous.

Queuing Effects: Why 70% Is the New 100%

 utilization ρ vs wait time explodes non-linearly:

 ρ=0.5 → modest queues        ρ=0.8 → wait ~4× service time
 ρ=0.9 → wait ~9×             ρ=0.95 → wait ~19×

 running components near saturation makes latency collapse
 LONG before throughput does. capacity plans that target
 "until CPU hits 100%" are already dead at 80%.
 
 rule: provision for peak ≤ ~60–70% utilization

The Analysis Loop

 measure → identify top constraint → fix cheapest lever for IT
 → re-measure (bottleneck moved?) → repeat

 tools in order of value:
 distributed traces   where time goes per request
 RED/USE dashboards   rate/errors/duration + utilization/saturation
 load tests           find the wall BEFORE production does

Interview Framing

“System is slow — what do you do?” tests method over guesses. Scored shape: demand traces/metrics first, walk a concrete request breakdown, name the dominant contributor, apply Amdahl to size expectations (“fixing the DB’s 80% share caps us near 4x even with heroic sharding”), then note the bottleneck migrates. Guessing “add Redis” without measurement is the anti-signal.