Error Budgets

The Organizational Problem It Solves

 every team has this fight:

 product:  "ship features! velocity!"
 sre/devs: "stabilize! we just paged all weekend!"

 both are right. arguments resolve by seniority or exhaustion.

 ERROR BUDGET: convert the argument into arithmetic.

   SLO 99.5% → budget = 0.5% of requests may fail
   = a SPENDABLE allowance of unreliability.

The Mechanics

 monthly window, SLO 99.5%, 30M requests:

 budget = 150,000 failed/slow requests

 spend sources (everything that consumes reliability):
 - incidents/outages        - risky feature rollouts
 - experiments gone wrong   - planned maintenance

 policy (the treaty):
 ┌───────────────────────┬────────────────────────────┐
 │ budget remaining      │ allowed behavior           │
 ├───────────────────────┼────────────────────────────┤
 │ > 50%                 │ full velocity, take risks  │
 │ 25–50%                │ normal caution             │
 │ < 25%                 │ freeze risky launches      │
 │ exhausted             │ reliability work only;     │
 │                       │ releases need approval     │
 └───────────────────────┴────────────────────────────┘

 the same policy binds BOTH sides. no more meetings about vibes.

Burn Rate: The Operational Signal

 budget spent per unit time vs sustainable pace:

 sustainable burn: 1.0× (budget lasts the full window)

 alerting on burn RATE, not raw errors:
   14.4× over 1h   → page (budget gone in ~2 days at this rate)
   6×   over 6h    → ticket (week-scale exhaustion)
   1×              → silence (normal life)

 multiwindow alerts cut false pages:
   fast-burn AND slow-burn both tripping = real incident;
   spike-and-recover = blip, ticket not page.

Policy Edge Cases Worth Deciding Early

 □ who can override a freeze? (named humans, documented)
 □ do PLANNED maintenance spend budget? (usually yes —
   it's real unavailability to users regardless of intent)
 □ shared vs per-team budgets for platform failures?
   (attribution fights are real; decide before incident)
 □ budget RECOVERY: only via window rolling forward +
   improved reliability. no manual resets. ever.

Interview Framing

“Product wants weekly deploys; ops says system too fragile” scored answer: error-budget treaty as THE resolution mechanism, concrete numbers walked through, graduated response table, burn-rate alerting mentioned, and the cultural point — budgets make risk-taking SAFE within measured bounds, aligning incentives instead of adjudicating meetings. This is an SRE-culture question wearing a design question’s coat.