Schema Registry

The Problem It Solves

 without a registry, schemas live... where? in code? hope?

 - producer ships breaking change → consumers crash in PROD
 - nobody knows who consumes topic X or which fields they read
 - "what does this 2023 event look like?" → archaeology

 SCHEMA REGISTRY: a service storing every schema version,
 enforcing compatibility rules AT REGISTRATION:

 producer ──register/check──► [registry] 
              │  compatible? store version ✓
              │  incompatible? REJECT before any harm ✓
              ▼
         publish message (schema id attached)

 consumer ──fetch schema by id──► deserialize safely

How It Works Day-to-Day

 produce path:
   serialize(event) with schema v12 → wire format:
     [schema_id=812][binary payload]    ← id inline!
   registry checked compatibility at v12 registration time.

 consume path:
   read id → fetch schema (CACHE IT locally!) → decode.
   consumers don't need the schema in advance — self-describing.

 compatibility modes per subject (topic):
   BACKWARD:          new schema reads old data
   FORWARD:           old schema reads new data  
   FULL:              both (default recommendation)
   NONE:              anarchy (don't)

Formats and Their Fit

Format	Strengths	Notes
Avro	mature registry support, evolution rules	JSON-defined; compact binary
Protobuf	cross-language, optional fields native	aliases aid renames
JSON Schema	human-readable, web-native	verbose on the wire

 all three work with registries. avro/kafka is the classic;
 protobuf gaining everywhere. pick one ecosystem-wide —
 mixed formats multiply tooling pain for zero benefit.

What Registries Actually Enforce

 the governance wins:

 □ COMPATIBILITY GATES: bad schemas rejected at CI/deploy,
   not discovered as production crashes ✓
 □ SCHEMA DISCOVERY: "what's in orders-topic?" answered
   authoritatively, with full version history
 □ CONSUMER IMPACT: usage metadata tracks which apps
   deserialize each schema → removal census automated
 □ DOCUMENTATION co-located: field descriptions travel
   with schemas, not in stale wikis

Operational Notes

 □ registry is CRITICAL PATH: unavailable registry can block
   producers/consumers → run HA, cache aggressively client-side
   (id→schema mappings are immutable — cache forever safely!)
 □ SUBJECT naming: topic-name vs record-name strategies;
   choose once, consistently
 □ STAGE promotion: dev → staging → prod schemas via same
   API; no hand-edits in prod. ever.
 □ EXPORT/backups: registry outage shouldn't mean archaeology.

Interview Framing

“How do 30 services share Kafka topics without integration hell?” scored answer: registry as enforcement point (compatibility at registration), wire-format-with-schema-id mechanics, mode selection (FULL default), the discovery/census benefits, HA+client-caching note. Mentioning that registry turns schema discipline from convention into infrastructure is the sentence that lands it.