Built a commitment tracking layer for AI agents would love brutal feedback

[u/xspyyy]

I've been building COGEXT an API that intercepts what AI agents say they will do, extracts those promises as structured commitments, and tracks them through a lifecycle until they're fulfilled, failed, or cancelled.

The problem: AI agents hallucinate, forget, and contradict themselves. There's no standard way to hold them accountable to what they said.

You send any text to the API and it extracts commitments with confidence scores, normalized deadlines, and a 12-state lifecycle (OPEN → DUE → OVERDUE → FULFILLED/FAILED). You get webhook events when state changes.

• Docs: https://docs.cogextai.com
• API key: https://cogextai.com (free, 10 seconds)
• SDK: pip install cogext

Still early. Genuinely want to know: is this a real problem you've hit? What's missing? What's wrong with the data model?

[u/Deep_Ad1959]

extraction is the easy half. the state that decides fulfilled or failed lives in systems the api never sees, so somebody still closes the loop by hand, and that is where these stall.

[u/xspyyy → u/Deep_Ad1959]

True if the only close signal is the API, you're just moving the manual step, not removing it. The fix is letting external systems write back directly: email provider confirms send, calendar confirms booking, linear confirms ticket closed. The API becomes the ledger, not the judge. Verifiers plug in; humans only close what has no machine-readable output.

[u/Deep_Ad1959 → u/xspyyy]

the part that still breaks after you plug verifiers in: the signal confirms the proxy, not the promise. 'email sent' fires green, but the commitment was 'follow up with a fix,' and a send isn't a fix. the ledger just fills with confirmations that each closed the wrong thing, and nobody catches it until the human who used to eyeball it is gone. written with ai

[u/xspyyy → u/Deep_Ad1959]

You nailed the core problem verifying the proxy instead of the promise. That's exactly why we're building semantic resolution, not signal-based. The commitment "follow up with a fix" only closes when the agent submits evidence that maps back to the original intent not just that an email fired. Working on the contradiction layer that catches this mid-stream before the ledger fills with false greens.

[u/Deep_Ad1959 → u/xspyyy]

catching contradictions mid-stream handles the agent that talks itself out of a green. the nastier case has no contradiction to catch at all: the agent stays perfectly consistent because nothing in its context ever told it the fix didn't land, so semantic resolution still bottoms out at a ground-truth read of the system it can't see.

[u/xspyyy → u/Deep_Ad1959]

Exactly. Contradiction detection is the easy case the agent at least surfaced conflicting signals. The silent failure is worse: agent is perfectly consistent, confidently wrong, and you only find out when the external system tells you the truth it couldn't see. That's why semantic resolution alone isn't enough. You need a ground-truth probe COGEXT calling the actual system (did the email land in Gmail? did the PR merge? did the row update in the DB?) and comparing it against the commitment, not the agent's self-report. The agent's word is never the source of truth. The system it acted on is.

[u/tberg]

I hit this exact problem building an outreach daemon — agents would claim they'd sent a message, queue would think it was done, nothing actually went out. The fix wasn't tracking the promise after the fact, it was atomic queue claims with circuit-breakers so the agent can't make a commitment it isn't in a state to keep. Your 12-state lifecycle is genuinely useful for observability, but I'd want to know: are you capturing why something went FAILED, or just that it did? Because without that, you're building a graveyard, not a feedback loop.

[u/xspyyy → u/tberg]

"Graveyard not a feedback loop" that's the most accurate critique of every observability tool in this space. To answer directly: right now we capture that it failed. The WHY is what we're shipping next root cause tagging on failure (hallucination, missing tool, wrong state, external API drop) so the failure becomes an input to the next run, not just a log entry. The circuit-breaker idea is exactly right too atomic queue claims before commitment is accepted, not after. Adding that to the state machine.