Structured Logging

String Logs vs Event Logs

 UNSTRUCTURED (the old way):
   "User 912 placed order 42 for $89.90 at 14:03"
   → greppable-ish; unqueryable. every consumer parses prose.

 STRUCTURED:
   {"ts":"14:03:22","level":"info","msg":"order_placed",
    "user_id":912,"order_id":42,"amount":89.90,
    "trace_id":"abc123","service":"orders"}

 fields stay FIELDS:
   filter: level=error AND service=orders AND amount>1000 ✓
   aggregate: p95 amount by user-tier ✓
   the log line becomes a QUERYABLE EVENT RECORD.

The Field Discipline

 standard envelope every event carries:

 □ ts (iso8601, UTC!), level, msg
 □ service, version, host/pod
 □ trace_id / span_id (the interlock!)
 □ correlation_id where business flows span async hops
 plus EVENT-SPECIFIC payload fields — named consistently:
   user_id not userId not uid (pick one; lint it)

 LEVEL semantics worth enforcing:
   ERROR: needs attention/action (paged or ticketed)
   WARN:  degraded but handled (monitored trends)
   INFO:  significant business/flow events (sparse!)
   DEBUG: diagnostic detail (off in prod by default)

 anti-patterns that ruin structured logging:
 ✗ string interpolation into msg with the data lost from fields
 ✗ logging inside loops ("processing item 1..2..3...N")
 ✗ PII/passwords/tokens in payloads (scrubbers + review)
 ✗ multi-KB dumps as "context" (link to object storage instead)

Sampling and Volume Control

 logs scale linearly with traffic — budget deliberately:

 □ TIER by value:
     errors/warns: always keep
     key business events: keep
     high-frequency info: sample (1-in-10) with counters kept
     debug: env-gated, auto-timeout switches
 □ RATE LIMITS per logger: a pathological loop can't emit
   infinite lines (circuit-break your own logging!)
 □ COST VISIBILITY: per-service log-volume dashboards;
   regressions reviewed like performance bugs

 sampling rule of thumb: if you can't remember why a log
 exists or when you last queried it — it's cost without signal.

Interview Framing

“Debug why orders fail for users over $1000 spend” scored shape: structured-event schema proposed with query shown answering it directly, field-discipline rules named (consistent ids, trace linkage), volume/sampling strategy included, PII-scrubbing mentioned unprompted. Logging questions test whether you treat logs as ENGINEERED DATA PRODUCTS — schema-first thinking is the tell.