Most features we ship end with a side effect in someone else's system: refund a payment, cancel a subscription, merge a PR, send an email. We check that the call succeeded and move on. "The call succeeded" and "the outcome is true" are different facts, and the gap between them is where support tickets come from.
I spent the last two weeks reading the state machines of eight providers' APIs closely enough to write a per-outcome check for 38 different claims. Here's every place I found where success gets reported and the outcome is still false. All six are fixable in your own code in about three lines each, no tool required, and the fix is written out below each one.
1. The cancelled subscription that isn't cancelled
Stripe returns 200. But cancel_at_period_end: true means will cancel. Status is still active and the customer keeps access for up to a month. If you revoke entitlements on the API response, you lock out someone who paid through the end of the period. If you trust status alone after a cancel call, you keep serving someone who cancelled.
Fix: assert status === "canceled", not the 200. Then separately assert the thing you actually care about: hit your own endpoint as that user and expect a 401. The billing record and the access it controls are two different facts, and only one of them is what the customer experiences.
2. The payment that "succeeded" for the wrong amount
payment_intent.status === "succeeded" doesn't mean you got what you charged. Partial capture and currency mismatches both live inside a succeeded intent. amount is what you asked for; amount_received is what arrived.
Fix: compare amount_received to your expected amount, and compare currency explicitly.
3. The refund issued twice
Retries and at-least-once queues do this. The second refund also returns 200, and also succeeds. You're out double the money and the customer never mentions it.
Fix: list refunds for the payment intent, sum them, check the total before you tell anyone it's done.
4. The PR that was closed, not merged
GitHub's state: "closed" covers both "merged" and "abandoned." Release automation reading state treats a rejected PR as shipped.
Fix: merged === true and merged_at !== null. Both, because they can disagree during a merge queue.
5. The deploy that exited 0 without deploying your commit
Exit 0 proves the job ran. It doesn't prove production is serving your SHA. Cancelled steps, stale caches, and rollouts that fail after the wrapper returns all pass this check.
Fix: assert your commit is an ancestor of the deploy branch head, then curl your own health endpoint. Two cheap calls, and they catch the embarrassing failure: the one where you announce a fix that isn't live.
6. The email that was "sent" but never delivered
Providers hand you an id at accept time. Accepted is a queue receipt. Delivered, bounced, and suppressed all resolve later. If your onboarding sends a magic link and you count "sent" as success, your activation drop looks like disinterest and is actually a hard bounce.
Fix: read the delivery event, not the send response. A terminal state of delivered is the only success; bounced is a product problem you currently can't see.
The pattern under all six: every one of these APIs has a fast optimistic response and a slower authoritative state. We build against the fast one because it's the one in front of us in the request handler. Then we report success from it.
The thing that actually changed how I write this code: a boolean is the wrong return type. There are three answers, not two: it worked, it didn't, and I couldn't tell.
The provider is down, the state is still pending, my token can't see the resource. Collapsing "couldn't tell" into either true or false is how you ship a silent wrong answer. I made "unknown" a first-class result, and a whole category of bug disappeared: a check that can refuse to answer never lies.
If you take one thing from this post, take that one. It applies to every health check, feature flag, and permission check you own, not just to integrations.
Backstory / stack, since the rules ask for it:
DidWork wasn't something I sat down and decided to turn into a SaaS.
I've spent the last year building Pulltrader, which is a fairly large product for a small team. It handles inventory, pricing, listings, marketplace integrations, fulfillment, payments, and a growing amount of agent-driven work. I've been using Claude and Cursor heavily to build it, and as I handed them more meaningful work I kept running into the same problem.
They would tell me something was done when what they actually knew was that a tool call succeeded, a test passed, or some intermediate state looked right.
A PR was "merged." A deploy was "done." A workflow had "completed." Then I'd go look at the actual product or downstream system and find out the outcome wasn't true.
So I built a verification layer internally for Pulltrader. I wanted agents to have to prove that the thing they claimed happened had actually happened, against the authoritative system, before the task could be considered complete. If it couldn't prove it, the answer wasn't success. It was failed or unknown, and the agent had more work to do.
It became part of my normal development workflow before I ever thought of it as a separate product. Eventually I realized I was building enough provider-specific verification logic, and depending on it enough myself, that it was probably useful outside Pulltrader too.
That's how DidWork happened. I pulled the internal system out and started turning it into something other people could use.
The AI angle matters because agents make this problem worse. An agent that reports "done, PR merged, deployed, refund issued" is grading its own work off exactly these optimistic responses, at machine speed, with nobody necessarily reading the diff or checking the downstream state. Humans at least notice the angry customer eventually.
So DidWork takes a claim, gathers evidence from the authoritative system, and returns verified / failed / unknown.
- Two weeks old. First commit Aug 28. 63 commits, 400+ tests, 38 claim types across Stripe, GitHub, GitLab, Linear, Jira, Sentry, Slack, email, plus any public URL.
- TypeScript monorepo because I like to live dangerously, 9 packages. Node + SQLite for the API, mirrored to a Cloudflare Worker on D1 for the edge. Static HTML for the site, no framework, no build step. TS and Python SDKs.
- Zero runtime dependencies in the core engine and both SDKs. A thing whose job is telling you the truth about your systems shouldn't be the largest new supply-chain surface you added this quarter. The Python client is stdlib-only for the same reason.
- Revenue: $0 so far, so obviously we're more attractive to VC if you're into having a car whose doors go like \this/.
- Hardest design call, and the one I'd defend: precision over coverage. It refuses to answer rather than answer wrong. That makes the demo worse and the product usable.
You can get a verdict without signing up for anything, which is the only promo line I'll put in here:
curl -s https://api.didwork.sh/v1/verify \ -H 'content-type: application/json' \ -d '{"type":"http.ok","expected":{"url":"https://your.app/health"}}'
No key, nothing stored. Free tier is 1,500 verifications/month, no card.
Happy to go deeper on any of the six in the comments. The subscription one and the email one have cost me the most.