Canary Deployment

The Pattern

 expose the new version to a tiny real cohort, VERIFY, expand:

 [v1: 99% of users]  [v2: 1% — canaries]

 the cohort hits v2's REAL code paths under REAL load:
 synthetic tests can't reproduce production's chaos —
 canaries borrow reality.

 ladder: 1% → watch gates → 5% → 10% → 50% → 100%
 each step: metrics compared against control (v1) cohort.

 rollback at any rung = instant weight reversion.
 blast radius of a bad build ≈ current exposure %. ✓

What Gates Must Watch

 compare CANARY vs CONTROL, not vs absolute thresholds
 (traffic mix shifts would fool absolutes):

 □ error rate delta (5xx, business errors)
 □ latency deltas p50/p95/p99 per endpoint class
 □ saturation on canary pods (they're fewer — watch hot spots)
 □ downstream impact: did v2 stress shared stores differently?
 □ BUSINESS signals where available: conversion, checkout
   completion (the metrics that pay for all this machinery)

 statistical honesty at small percentages:
   1% of low-traffic endpoints = few samples = noisy gates.
   either wait for sample volume or gate on request-count
   floors, not just time windows. premature promotion from
   empty data is the classic canary failure.

Cohort Selection Strategies

Strategy	Mechanism	Use
random %	per-request dice	default
sticky hash by user	same user always v2 during window	session coherence
internal-first	employees/allowlist	dogfooding phase
attribute-based	region/plan/device	targeted risk

 sticky-by-user matters more than teams expect:
 a user bouncing between versions mid-flow sees inconsistent
 behavior (different feature flags, cache formats) —
 hash-bucketing keeps experiences coherent AND makes
 support triage possible ("which version were you on?").

Canary vs Its Cousins

 CANARY:      gradual %, automated gates, fast rollback
 BLUE-GREEN:  atomic switch, no mixed state, costly footprint
 ROLLING:     batch replacement, simplest, weaker gating
 FEATURE FLAG: deploy≠release; complements ALL of them

 they compose: canary-rollout infrastructure + flags inside
 the binary = independent dials for INFRASTRUCTURE risk
 and PRODUCT risk. mature shops run both deliberately.

Interview Framing

“Roll out a risky rewrite to millions of users” scored shape: staged ladder with control-vs-canary comparison framing, gate-metric list including business signals, sample-size honesty at small percentages, sticky-cohort rationale, composition-with-flags note. Canary questions are everywhere because they test STATISTICAL reasoning plus operational empathy in one answer — most candidates forget the noise floor; don’t be most candidates.