Bulkheads

The Shared-Pool Failure

 one thread pool / connection pool / memory budget shared by all:

 [api service]
   threads(200) ── serve: search, checkout, reports, images

 report-generation job goes rogue:
   eats 190 threads → checkout requests QUEUE →
   revenue path DOWN because a BATCH JOB misbehaved.

 no isolation = everything shares everything =
 any component's pathology is everyone's outage.

The Pattern

 partition resources per workload class:

 [api]
   pool-A: 40 threads  → critical: checkout, payment
   pool-B: 80 threads  → standard: search, browse
   pool-C: 20 threads  → bulk: reports, exports
   pool-D: 10 conns    → legacy integration

 report job melts? pool-C saturates ONLY.
 pools A/B untouched. checkout sails through ✓

 isolation dimensions (apply several):
 - THREAD/worker pools per dependency or endpoint-class
 - CONNECTION pools per downstream database/service
 - MEMORY quotas per tenant/plugin (container limits!)
 - DISK/IO quotas per subsystem
 - K8S RESOURCE LIMITS per pod = the macro-bulkhead

Sizing the Compartments

 each pool sized from ITS OWN demand math:

 pool-A (checkout): peak 200 rps × 50ms avg = ~15 concurrent
                    + burst headroom → 30 threads
 pool-C (reports):  whatever remains; can queue/slow happily

 principles:
 □ critical paths get GUARANTEED capacity first
 □ best-effort work gets the leftovers + queuing
 □ total ≤ machine capacity (no overcommit illusion)
 □ saturation of a pool is VISIBLE and alertable
   (queue depth, wait time per pool — not just CPU!)

 wrong-sized bulkhead = self-inflicted starvation;
 review sizing when traffic mix shifts.

Without bulkheads	With
noisy neighbor kills all	contained to its compartment
one slow dep exhausts pool	only that dep’s pool fills
debugging “who ate the threads?“	per-pool attribution instant

Bulkheads in the Wild

 - netflix/hystrix-style thread-pool & semaphore isolation
 - database: separate users/roles with connection limits
   (analytics can't consume OLTP's connections)
 - message consumers: per-tenant concurrency caps
 - browser: HTTP/1.1's per-host connection limit was a bulkhead!
 - kubernetes: requests/limits + network policies +
   node pools per workload class

Interview Framing

“A runaway analytics job took down checkout” scored shape: name missing bulkheads as root cause, draw pool-per-workload-class diagram with numbers, sizing-from-demand shown for the critical pool, monitoring-per-pool added, k8s-limits as macro version mentioned. The phrase “guaranteed capacity for critical paths” is the design philosophy in five words — use it.