Circuit Breaking in the Mesh

Two Mechanisms, One Goal

 meshes offer related-but-distinct protections:

 1. OUTLIER EJECTION (per-endpoint health):
    consistently-failing ENDPOINT benched from the pool.
    (load-balancing lesson's passive health.)

 2. CIRCUIT BREAKING (capacity protection):
    limits on CONCURRENT demand toward a cluster;
    excess rejected FAST instead of queueing into doom.

 [sidecar]──► cluster: reviews
   maxConnections: 100      ← tcp pool cap
   maxPendingRequests: 50   ← queue cap
   maxRequests: 200         ← in-flight cap (http2)
   maxRetries: 30           ← retry storm ceiling

 caps hit → immediate local failure response → caller falls
 back. the dependency is shielded from stampedes; callers
 degrade fast instead of hanging slow.

What Mesh Breaking Does NOT Do

 classic breaker semantics (open→cooldown→probe) are NOT
 the default mesh behavior — know what you configured:

 mesh caps = LOAD SHEDDING at concurrency boundaries.
 they prevent resource exhaustion; they don't auto-open/close
 on error RATES by themselves.

 error-rate-driven behavior comes from:
 - outlier ejection (removes bad endpoints progressively)
 - your own app/mesh-policy layering (retry budgets +
   fallback routes on errors)

 practical stack for a sick dependency:
   ejection benches dying instances ✓
   concurrency caps prevent stampede ✓
   retry budget prevents multiplication ✓
   route fallback serves cached/default ✓
 four policies composing = the real circuit breaker.

Configuring Without Footguns

Knob	Guidance
maxConnections	≈ peak concurrent × headroom per instance
maxPendingRequests	small! queues hide sickness
maxRetries	fleet % budget; tiny relative to traffic
ejection base time	start 30s; escalate on repeat

 sizing sins seen in production:
 ✗ pending-queues of thousands (latency bombs)
 ✗ caps sized from CPU math ignoring downstream DB pools
 ✗ no per-cluster differentiation: one global config for
   both the auth service and the batch exporter
 ✗ forgetting HTTP/2 multiplexes requests over fewer
   connections → maxConnections nearly irrelevant there;
   maxRequests is the live knob

Interview Framing

“Configure protection so a degrading reviews service can’t take down checkout” scored shape: distinguish ejection-vs-concurrency-caps explicitly (the common conflation), show the four-policy composition stack, HTTP/2 knob subtlety named, queue-small principle stated with reasoning. The question tests whether you configure meshes from FAILURE MODELS or copied YAML — explain why each number exists.