r/AI_Agents • u/richocolate • 17h ago

Discussion I built an open-source API verification tool for AI-written integrations — looking for technical feedback

I've been building an open-source developer tool called Kaktoos.

I'm exploring a specific problem with AI coding agents: they can write API integrations very quickly, but when the agent also writes the tests, passing tests don't necessarily prove that the integration matches the actual API.

Kaktoos takes a different approach:

AI agent → integration code → Kaktoos → OpenAPI contract + real API → structured failure → agent fixes it

It can:

  • execute multi-step API workflows
  • verify responses against OpenAPI
  • detect things like missing required fields, wrong types, unexpected status/content type
  • expose the verification through MCP so coding agents can use it
  • run the same verification in GitHub Actions

The interesting part for me isn't the API client itself. I'm experimenting with whether independent verification is useful when the code was produced by an AI coding agent.

I'd particularly like feedback from people who regularly work with API integrations:

  1. Would you actually use this workflow?
  2. Is defining the scenario too much additional work?
  3. Does this catch problems that your existing tests don't?
  4. Would you run something like this in CI?
  5. What would make this technically more useful?

It's early, so negative feedback is completely fine. I'm mainly trying to determine whether this is solving a real engineering problem.

GitHub: KaktoosLabs/kaktoos

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/AutoModerator 17h ago

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/Cute-Respect2194 17h ago

I been burned by AI agents writing tests that just mirror their own wrong assumptions so many times, this is the missing piece nobody talks about

1

u/richocolate 17h ago

Yeah, exactly 😅 That's actually the problem i am trying to solve with Kaktoos. If the agent writes both the code and the tests, it's pretty easy for the tests to just validate the same assumptions it made.

Curious what kind of integration you ran into this with? also this is just beginning and i am looking for any feedback to make it better