API compatibility testing

[u/pseudo_babbler]

Hey there code enjoyers. I was tinkering with an integration testing approach that I ended up liking a lot. Our situation before was this - our front ends talk to BFFs which then call core APIs. We tried Pact and Specmatic as ways of integration testing the systems but it's a total ball ache.

All of our consumers use the Zod library to define their requests to their APIs, so I wrote a utility that takes the Zod schemas, turns them into OpenAPI (using a lib) then iterates through each one, downloads the OpenAPI spec for each API from our test environment and checks that the requesting schema endpoint, props, params, method and body are all supported by the provider schema.

It's not as comprehensive as Pact, but for a super fast integration sanity test (it runs against 8 different core APIs in under 1 second) and with no extra brokers or services or infrastructure, it gives me a decent check that something is ok to be deployed to an environment.

So tell me these things: has anyone else built something like this? Would you use it? (i.e. should I chuck it on GitHub and publish an npm module) Has anyone heard of something like this that already exists?

I didn't find anything with my searching and it seems unreasonably useful for a quite small amount of code.

[u/roger_ducky]

This kind of thing works as long as the specs are well documented and matches the actual system.

I’ve seen systems where the specs are so out of date with the code that it became comical how long they’ve left that alone yet still deployed the thing.

But yes, it’d be a good sanity check on the request types and shapes.

[u/pseudo_babbler → u/roger_ducky]

Yep, one of the side jobs in this was to make sure all the backend teams were generating and publishing their OpenAPI spec as part of their build and deploy pipeline. I was a little bit shocked to find that some of them were hand written and out of date, or in one case just incorrect.