Load Balancing in the Mesh

Where the Decisions Happen

 every request, the caller's sidecar picks an endpoint:

 [app req]→[sidecar]──choose endpoint──►[peer]
              │
        policy stack:
        1. subsets? (version labels filter pool)
        2. locality priority (zone→region→anywhere)
        3. algorithm within tier (RR/LEAST_REQUEST/RING_HASH)
        4. healthy? (active checks + passive ejection)

 vs kube-proxy round-robin-ish: mesh LB is PER-REQUEST,
 POLICY-RICH, and OBSERVABLE per decision.

The Algorithm Menu

Algorithm	Behavior	Best for
ROUND_ROBIN	even rotation	homogeneous default ✓
LEAST_REQUEST	fewest in-flight wins	skewed latencies
RING_HASH / MAGLEV	consistent hashing by key	sticky sessions/cache affinity
RANDOM	cheapest, statistically fine	huge pools
EWF (weighted)	percentage splits	canaries/migration

 consistent hashing deserves its note:
 RING_HASH on header/cookie → same user → same backend
 (sticky caches/sessions) WITHOUT session-store coupling.
 but watch hot keys + pool changes causing reassignments.

Locality-Aware Routing

 the cost+latency optimizer:

 priority ladder per request:
   prefer same ZONE endpoints
   → exhausted/unhealthy → same REGION
   → finally cross-region

 weighted spill-over tuning:
   distribute: zone: 80% local, 20% remote
   = smooth failovers without full-zone cliff

 why teams turn knobs here:
 - cross-zone bandwidth is BILLED (real money at scale)
 - zone-local RTT ~1ms vs multi-ms cross-zone
 - zone-failure tests verify actual spill behavior!
   (misconfigured locality = silent all-cross-zone traffic)

Passive Health: Outlier Ejection

 the mesh watches ACTUAL results and benches losers:

 - consecutive 5xx from endpoint X → eject from pool for N sec
 - successive ejections → longer bans (cumulative penalty)
 - max-ejection % caps protect tiny pools (never eject 100%)

 [pool: A B C]──C errors repeatedly──►[A B] serving
                        C benched 30s → probe back later

 pairs with ACTIVE health checks (configured probes):
 active removes dead-fast; passive catches slow-broken
 (timeouts, error spikes) that checks miss.

Interview Framing

“Recommendation service: some instances slow after cache expiry storms — how does the mesh cope?” scored shape: LEAST_REQUEST + outlier-ejection combo proposed with mechanics, locality-ladder explained including bill rationale, consistent-hash sticky option named if cache-affinity helps, active/passive health pairing stated. LB questions grade whether you see balancing as a per-request POLICY STACK rather than a knob labeled “round robin.”