Cardinality

The Multiplication Trap

 each unique LABEL VALUE COMBINATION = one stored time-series:

 http_requests_total{
   service="orders",        1 value
   route="/orders/:id",     50 routes
   method="GET|POST",       ×2
   status="200|400|500",    ×3
   region="us|eu|ap"        ×3
 }
 → 1×50×2×3×3 = 900 series. fine.

 now add user_id: ×10M users → 9 BILLION series. ✗✗
 monitoring backend dies; dashboards blank DURING INCIDENTS;
 costs explode. cardinality is THE operational limit of
 metric systems (prometheus-class especially).

Budgeting Like an Engineer

 treat series counts as capacity to be managed:

 □ PER-SERVICE BUDGETS: hundreds, not thousands; alert at 80%
 □ HIGH-CARDINALITY LINT in CI: reject user-id/email/raw-path
   labels automatically — humans forget; linters don't
 □ URL NORMALIZATION mandatory:
     /users/912/orders/88  →  route=/users/:id/orders/:id
   unbounded paths are infinite labels
 □ EXPLORE ELSEWHERE: "per-user latency" questions belong in
   TRACES (sampled) or LOGS (targeted), never metric labels

 the exemplar escape hatch:
 metrics keep low-cardinality aggregates; sampled traces LINKED
 from dashboard panels give per-request depth on demand.
 best of both without the explosion.

Where Cardinality Sneaks In

Innocent-looking label	Actual cardinality
path=“/api/…” raw	unbounded (ids in urls)
error_message	every distinct string!
pod_name	bounded-ish but churn-heavy
experiment_variant × feature_flag combos	product-team multiplication
customer_id “just temporarily for debugging”	forever

 the debugging-label trap deserves its warning:
 someone adds customer_id "for a week to catch a bug" —
 then the series exist FOREVER until manually hunted.
 temporary high-cardinality needs: logs + trace exemplars,
 with time-bounded dedicated tooling if truly required.

Operating With the Constraint

 □ AGGREGATE UP, drill via links: global → regional → instance
   as separate series; dashboards start aggregated
 □ PRE-AGGREGATION pipelines for known-hot dimensions:
   recording rules compute common queries cheaply
 □ CARDINALITY REVIEWS quarterly: top-10 heaviest metrics,
   prune dead ones (dead series accumulate silently)
 □ BACKEND CHOICE informed by it: prometheus-class strict;
   some TSDBs tolerate more at higher cost — still budgeted,
   never unlimited

Interview Framing

“Our monitoring falls over during incidents” scored shape: multiplication mechanics taught with the concrete math example, lint-and-budget enforcement named, URL-normalization rule demonstrated, exemplar-based exploration offered as the escape hatch, sneaky-sources table condensed. Cardinality questions test whether you’ve OPERATED metrics at scale — everyone adds labels; operators know which labels kill.