Identify Bottlenecks

The Definition That Matters

A bottleneck is the component that limits total system throughput — traffic piles up in front of it while neighbors idle. Like highway congestion, widening anywhere else changes nothing:

        5k rps          1k rps ceiling        5k rps
 Client ─────► App tier ─────► Single DB ─────► Cache
               (fine, N nodes)  ▲
                                │ THE bottleneck:
                                everything queues here;
                                app nodes sit idle waiting

Throughput of the chain = throughput of its narrowest pipe. Finding and widening that pipe is the entire exercise.

The Usual Suspects

Bottleneck class	Typical form	First symptom
Single-writer database	All writes serialize on one primary	Write latency climbs with load; replicas idle
Hot key/hot partition	One cache node or shard gets 10% of keys	Uneven CPU across symmetric fleet
Connection exhaustion	App pools × nodes exceed DB limit	”too many connections”; pool waits
Synchronous fan-out	One request blocks on 10 downstream calls	p99 explodes with any dependency hiccup
Chatty internals	N+1 query patterns	DB QPS 10x user QPS
Lock contention	Row/counter hot updates	Threads stack on one row

Notice most are data-layer or coordination bottlenecks — stateless tiers scale trivially; shared state is where systems choke.

Queueing Intuition

Latency degrades non-linearly as utilization rises — this is why the 70% rule exists:

 utilization → latency multiplier (M/M/1 intuition)

 50%   ≈ 2x service time     calm
 70%   ≈ 3.3x                noticeable
 85%   ≈ 6.7x                SLOs breaking
 95%   ≈ 20x                 outage-shaped

 a component at 50% capacity is not half-wasted;
 it is the headroom that keeps p99 alive at peaks

Bottleneck analysis therefore targets components whose peak utilization approaches 70–80%, not averages.

Finding Them: Prediction, Then Evidence

Predict by walking the request flow asking “what saturates first at 10x?” — usually one of: primary DB writes, hottest cache, fan-out aggregator.

Confirm with evidence, never assumption:

Signal source	What shows
Latency percentiles per dependency	Which leg owns the tail
Utilization per component at peak	Who approaches the cliff
Queue depths / pool wait times	Where demand exceeds service rate
Load test at 2x current peak	The failure you’d meet next quarter

The load test is the honest oracle: it converts theory into observed collapse points.

Fixing Follows a Ladder

Once confirmed, widen in cost order:

 1. reduce demand      cache hits, batching, trimming chatty calls
 2. split the work     read replicas, functional partitioning
 3. add parallelism    more nodes behind LB (stateless only)
 4. divide the data    sharding — the expensive, last-resort widening
 5. shed load          deliberate degradation when all else lags spikes

Jumping to step 4 before exhausting 1–3 is how premature complexity happens.

Interview Framing

“Where’s your bottleneck?” is a standing interview question from the moment a diagram exists. Scoring reflex: point at the single-writer DB or the fan-out path immediately, justify with the numbers already estimated (“600 matches/sec against one Postgres writer puts us near the cliff”), and name the fix ladder rung. Bottleneck talk is where estimation pays rent.