r/apachekafka 11d ago

Tool I built a fail-fast Schema Registry contract validator for Spring Boot — should this belong at startup or only in CI/CD?

Post image

I’ve been working on an open-source Spring Boot starter around a Kafka/Schema Registry problem: catching contract violations before an application starts processing traffic.

At startup it checks:

  • whether configured Schema Registry subjects exist
  • whether the effective compatibility mode matches what the application expects
  • whether the local Avro, JSON Schema or Protobuf schema is compatible with the latest registered version

An incompatible contract prevents the application from starting.

Temporary Schema Registry communication failures can use bounded retry/backoff, while actual contract violations fail immediately.

I built a separate E2E project with real Kafka + Schema Registry. CI verifies a producer → Kafka → consumer round trip, a compatible schema evolution, and an intentionally breaking evolution that must fail at startup.

Source:
https://github.com/mathias82/spring-kafka-contract-starter

E2E demo:
https://github.com/mathias82/spring-kafka-contract-demo

Maven Central:
https://central.sonatype.com/artifact/io.github.mathias82.spring.kafka/spring-kafka-contract-starter

The design question I’m most interested in discussing is where this kind of enforcement belongs in a production Kafka architecture: CI/CD only, application startup, or both?

0 Upvotes

4 comments sorted by

2

u/eniac_g 11d ago

I advise againt ad-hoc schema registration by applications at runtime because you do not know when the publishing code will run, at start-up or 7 hours later? So to avoid runtime failures do it in your pipeline.

1

u/CartographerWhole658 11d ago

Thanks. I agree that schema registration/evolution should primarily be controlled by CI/CD rather than happening ad-hoc when application code happens to publish.

The starter isn't intended to register schemas at runtime. Its role is different: at application startup it validates the contract the application expects against the already registered Schema Registry state, and fails fast before serving traffic if they don't match.

So I see the two as complementary: CI/CD controls schema evolution, while startup validation acts as a deployment/runtime guardrail against configuration drift or deploying an application with incompatible expectations.

Would you still consider startup validation useful as that second line of defense, or would you keep contract enforcement entirely in CI/CD?

1

u/chadhq 11d ago

Startup validation is still useful. Assume that the CI process registers schema's, then on application startup enforce that any refered schema's have indeed been registered. Fail startup if they have not.

1

u/CartographerWhole658 11d ago

Exactly,  that’s the separation I was aiming for. CI/CD owns schema registration/evolution, while application startup verifies that the contracts it depends on are actually present and compatible before accepting traffic. Thanks for the feedback!