Backpressure

The Firehose Problem

 unbounded queues hide overload until they don't:

 producers write faster than consumers process:
 [producers] ██████████████► [queue: ∞] ──slow──► [consumers]

 queue grows silently → memory climbs → latency (through
 the queue) climbs → OOM or multi-hour staleness discovered late.

 BACKPRESSURE: push the "I'm full" signal UPSTREAM so
 production rate obeys consumption reality.

 [producers] ──►(429/credit-limited)──► [bounded queue] ──► [workers]
      slow down ◄──────────────────────────┘

Mechanisms by Layer

 SYNCHRONOUS calls:
   HTTP 429 + Retry-After; gRPC RESOURCE_EXHAUSTED
   caller's own backoff spreads load naturally

 QUEUES:
   BOUNDED queues: full → produce fails/awaits
   consumer-side: prefetch/count limits (rabbit basic.qos;
   kafka max.poll.records) — take only what you can chew
   
 STREAM PROCESSING:
   reactive-streams demand signals; flink credit-based flow;
   kafka pause/resume per partition when processing lags

 TCP had this from day one (flow-control windows) —
 application layers keep relearning the lesson above it.

Choosing Your Response to Fullness

 when downstream is saturated, upstream options:

 1. SLOW DOWN:   throttle producers (rate limit, credits)
 2. BUFFER:      bounded queue — with explicit bounds!
                 sized = acceptable delay × throughput
 3. SHED:        drop low-priority items (count them!)
 4. PERSIST:     spill to disk/object store, drain later
 5. FAIL:        error to origin (last resort, visible)

 anti-patterns:
 ✗ unbounded in-memory buffering ("it'll catch up")
 ✗ silent dropping (data loss disguised as backpressure)
 ✗ blocking caller threads indefinitely (cascades upward)

 pick per data value: telemetry sheds; orders persist.

Backpressure vs Load Shedding

	Backpressure	Shedding
Signal direction	upstream slowdown	local refusal
Data fate	delayed, not lost	dropped by policy
Complexity	coordination	classification
Best for	internal chains, valued data	edge overload, cheap traffic

 they compose: shed the worthless at the edge,
 apply backpressure for the rest through the pipeline.

Interview Framing

“Ingest service receives 100k events/s but processes 60k/s” scored shape: refuse unbounded-queue design immediately, present bounded-buffer + producer-slowdown mechanics (429/retry-after), classify overflow handling by event value (shed telemetry / persist orders), monitor queue-depth as THE saturation signal. Saying “the queue must be bounded and its bound is a designed number” is the senior tell.