Mesh Retries

Retries as Policy, Not Code

 mesh moves retry logic from every client library to
 declarative config on the route:

 apiVersion: networking/v1 (istio-flavored pseudo)
 route: /reviews
   retries:
     attempts: 2
     perTryTimeout: 2s
     retryOn: 5xx,reset,connect-failure

 effects fleet-wide instantly; app code stays dumb.
 behavior UNIFORM across languages — the drift problem
 that motivated meshes, solved for retries specifically.

The Mesh-Specific Semantics

 PER-TRY TIMEOUT is the star setting:

 without it: overall timeout 8s; attempt hangs 8s →
             zero time left → no actual retry happens!
 with it:    attempts get 2s each: try1(2s)→try2(2s)→
             maybe try3 within budget ✓ real retries occur

 RETRY-ON conditions worth knowing:
 - connect-failure/reset: safe transport-level cases
 - 5xx: status-based (careful: retried POSTs!)
 - gateway-error: 502/503/504 class
 - retriable-status-codes: custom list
 - retriable-status-codes + idempotency discipline

 REMEMBER-THE-BASICS still applies:
 - idempotency before retrying writes (always)
 - total budget = caller deadline envelope (arithmetic!)
 - jitter inherent in distributed timing but storms still
   possible under synchronized load spikes

Amplification Guardrails in Meshes

 mesh-level tools against retry storms:

 GLOBAL retry budget (envoy): cap total retries as %
 of requests fleet-wide — when upstream health degrades,
 retries auto-throttle ✓

 HEDGE policy (for tail latency, use sparingly):
   issue duplicate AFTER delay if no response;
   respond-with-first-wins. costs multiplier load —
   reads-only territory.

 pairing rule from the resilience lessons, mesh edition:
   retries handle TRANSIENT blips;
   outlier ejection handles consistently-bad endpoints;
   circuit breaking handles sustained dependency sickness.
   configure all three or misconfigure by omission.

Setting	Typical start
attempts	2 (total tries = 3)
perTryTimeout	p99 of healthy path
retryOn	connect-failure,reset (+5xx for GETs)
global budget	10–20%

Interview Framing

“Standardize retry behavior across 200 polyglot services” scored shape: mesh-policy approach stated with sample config semantics, per-try-timeout subtlety explained (the classic silent failure), amplification guardrails named (budgets!), idempotency precondition restated, three-tool division (retry/eject/break) mapped. This question checks whether you see mesh retries as a SYSTEM property requiring guards — not just YAML convenience.