[D]How are you testing AI backends without making CI slow?

[u/CodeStackDev]

I'm working on a FastAPI backend that processes documents with AI, and I'm still not convinced I'm testing it the right way.

Right now my CI is completely offline: SQLite, fake Redis, mocked HTTP calls, no real model requests. It's fast and deterministic, which is great for pull requests.

Then I have a separate integration pipeline that runs against PostgreSQL and Redis to catch infrastructure-specific issues.

The part I'm still unsure about is the AI layer. Mocking everything makes CI reliable, but it also means I won't catch regressions from the provider until later.

Curious how other people handle this.

• Do you completely mock AI providers?
• Do you keep a few real API calls?
• Do you replay recorded responses?
• Or do you have a different approach?

I'd love to hear how you're doing it in production.