---
title: "Schema Registry for AI Events: Confluent vs Karapace, Avro vs Protobuf vs JSON Schema"
description: "Without a schema registry, your AI event consumers break every time the producer adds a field. Confluent or Karapace plus Avro/Protobuf/JSON Schema gives you compatibility checks, evolution, and zero-surprise rollouts."
canonical: https://callsphere.ai/blog/vw4c-schema-registry-confluent-karapace-ai-events
category: "AI Infrastructure"
tags: ["Schema Registry", "Avro", "Protobuf", "Karapace", "Schema Evolution"]
author: "CallSphere Team"
published: 2026-05-07T00:00:00.000Z
updated: 2026-05-07T16:13:29.060Z
---

# Schema Registry for AI Events: Confluent vs Karapace, Avro vs Protobuf vs JSON Schema

> Without a schema registry, your AI event consumers break every time the producer adds a field. Confluent or Karapace plus Avro/Protobuf/JSON Schema gives you compatibility checks, evolution, and zero-surprise rollouts.

> **TL;DR** — Schema registry is the boring infrastructure that prevents 90% of event-driven outages. Confluent Schema Registry is the canonical implementation; Karapace is the API-compatible Apache 2.0 alternative. Pair it with Avro, Protobuf, or JSON Schema and your AI event producers and consumers evolve independently without a war room.

## The pattern

Producer team adds a field. Consumer team didn't get the memo. Production breaks. The fix: a schema registry between producers and consumers that checks every new schema against compatibility rules (BACKWARD, FORWARD, FULL). Producers can't ship an incompatible schema; consumers know the schema by reference instead of guessing from bytes.

## How it works (architecture)

```mermaid
flowchart LR
  Prod[Producer
writes Avro/Proto/JSON-SR] -->|register| SR[(Schema Registry
Confluent or Karapace)]
  SR -->|schema id 42| Prod
  Prod -->|magic byte + id 42 + payload| K[(Kafka)]
  K --> Cons[Consumer]
  Cons -->|GET schema 42| SR
  SR --> Cons
  SR -.compat check.-> Block[Reject incompatible schema]
```

Each event carries a 4-byte schema ID prefix. Consumers fetch and cache the schema by ID. Compatibility rules ensure new schemas don't break old consumers (BACKWARD) or new consumers don't break on old data (FORWARD).

## CallSphere implementation

CallSphere uses Karapace (open-source, Apache 2.0) for our internal Kafka topics across [Real Estate OneRoof](/industries/real-estate), Healthcare, IT Services, Salon, After-hours, and Sales. We picked Avro because compactness matters at our event volume. The CloudEvents envelope (post #12) wraps Avro-encoded `data` referenced via `dataschema`. After-hours and the simpler Bull/Redis paths use JSON Schema for readability. CI rejects PRs that change a schema without a compat declaration. 37 agents · 90+ tools · 115+ DB tables · 6 verticals · pricing $149/$499/$1499 · [14-day trial](/trial) · [22% affiliate](/affiliate). Browse [/pricing](/pricing) or take a [demo](/demo).

## Build steps with code

1. **Pick registry**: Confluent (managed, paid for SLA) or Karapace (open source).
2. **Pick format**: Avro (compact, schema evolution shines), Protobuf (gRPC alignment), JSON Schema (human-readable).
3. **Register schemas in CI**, not at runtime.
4. **Set compatibility level** per subject: BACKWARD by default.
5. **Producers serialize** with schema-aware serdes.
6. **Consumers deserialize** lazily via schema ID lookup + cache.
7. **Lifecycle**: deprecate old fields with default values; never remove.

```python
from confluent_kafka import Producer
from confluent_kafka.serialization import SerializationContext, MessageField
from confluent_kafka.schema_registry import SchemaRegistryClient
from confluent_kafka.schema_registry.avro import AvroSerializer

schema_str = """
{
  "type": "record",
  "name": "CallCompleted",
  "namespace": "ai.callsphere",
  "fields": [
    {"name": "callId", "type": "string"},
    {"name": "durationSec", "type": "int"},
    {"name": "outcome", "type": "string"},
    {"name": "verticalId", "type": ["null","string"], "default": null}
  ]
}
"""

sr = SchemaRegistryClient({"url": "http://karapace:8081"})
ser = AvroSerializer(sr, schema_str)
p = Producer({"bootstrap.servers": "kafka:9092"})

p.produce(
    topic="call.completed",
    key="abc",
    value=ser({"callId": "abc", "durationSec": 142, "outcome": "booked",
               "verticalId": "real-estate"},
              SerializationContext("call.completed", MessageField.VALUE)),
)
p.flush()
```

```bash
# CI compat check via REST
curl -X POST -H "Content-Type: application/json" \
  --data '{"schema": "...", "schemaType": "AVRO"}' \
  "http://karapace:8081/compatibility/subjects/call.completed-value/versions/latest"
```

## Common pitfalls

- **No registry** — every consumer guesses; outage at the next field add.
- **Registering at runtime** — race between producer and consumer; CI is the only safe place.
- **NONE compatibility** — defeats the point.
- **Avro defaults missing** — backward compat fails on field add.
- **Karapace + Confluent client mismatches** — Karapace doesn't implement every normalization feature; test in CI.

## FAQ

**Confluent vs Karapace?** Karapace is API-compatible and Apache 2.0; Confluent is managed and paid. Pick by ops appetite.

**Avro vs Protobuf vs JSON Schema?** Avro for compactness + evolution; Proto for gRPC alignment; JSON Schema for readability and JSON-native pipelines.

**BACKWARD vs FORWARD?** BACKWARD: new schema readable by old consumers. FORWARD: old schema readable by new consumers. FULL: both. Default to BACKWARD.

**How does CallSphere expose schemas?** Internal — but our outbound webhooks reference public CloudEvents `dataschema` URLs. See [/pricing](/pricing) and [/demo](/demo).

**Does it work for non-Kafka?** Karapace is Kafka-flavored, but you can use schemas anywhere via REST.

## Sources

- [Confluent Schema Registry Docs](https://docs.confluent.io/platform/current/schema-registry/index.html)
- [Karapace (Aiven-Open) on GitHub](https://github.com/Aiven-Open/karapace)
- [Schema Evolution and Compatibility (Confluent)](https://docs.confluent.io/platform/current/schema-registry/fundamentals/schema-evolution.html)
- [Schema Evolution in Avro, Protobuf, and JSON Schema](https://www.javacodegeeks.com/2025/06/schema-evolution-in-apache-avro-protobuf-and-json-schema.html)

---

Source: https://callsphere.ai/blog/vw4c-schema-registry-confluent-karapace-ai-events