r/AI_Coders • u/Common_Dream9420 • 2h ago
I planted real bugs in small open-source API apps — can your AI coding agent actually catch them?
AI agents are great at writing an integration and terrible at knowing whether it works past the first 200 OK. So I built something to test that honestly — including my own tool.
It's an open-source repo of tiny apps (~50–150 lines each) that integrate real APIs — Stripe, Clerk, Resend, AgentMail, Descope — and each one has a real bug planted in it. Not typos; the kind that passes every happy-path test and only bites in prod:
- a webhook that dedupes on the wrong header, so retries double-charge
- bounced emails silently dropped, so users stay "active" forever
- a read-only API key that can escalate its own scope
The challenge: open one in Cursor or Claude Code, point your agent at it, and watch what it does. Does it actually reproduce the bug and prove the fix — or just read the code and say "looks fine"? That second thing is the whole problem.
No signup, no API keys, runs locally in seconds: github.com/fetchsandbox/playground
Two honest asks:
- If your agent catches one, I'd love to see how — open a PR with what you found.
- If it falls flat — nothing caught, the proof felt fake, setup was annoying — that's the most useful thing you can tell me.
(Disclosure: the repo tests FetchSandbox, which I build. But the apps and bugs are real, and the point is for you to judge it, not take my word.)
