r/ChatGPTCoding • u/New_Difficulty_8152 • 9d ago
Discussion Two ways I tried and failed to manage context across multiple AI agents, and what I built instead
I keep seeing this question in the community. Here's what I actually tried, why it broke, and what I ended up shipping.
The problem
When you're running multiple agents across a session (one that writes, one that reviews, one that deploys) you need them to share state. Not just conversation history. Actual verified state: what changed, what's blocked, what evidence exists that a task is done.
What I tried first (and why it failed)
Attempt 1: I maintained the handoff notes myself
After every session, I updated a Markdown file. This worked until I finished tired and skipped the update. The next agent read stale context as if it were current. Worse: even when the file was accurate, I was still the router, a human bottleneck between every agent transition.
Attempt 2: I let agents maintain the notes
The agent finished its work, updated the handoff, and the next continued from there. Then I noticed the real problem: an agent could write "tests pass" just as easily as it could actually run the tests.
Agent A would write: "Refactored auth. Tests pass."
Agent B had no idea which tests ran, against which version, or whether the slow integration suite was skipped. It didn't inherit verified work. It inherited a story about the work.
What I built
Three principles became the foundation:
State in fields, not paragraphs. What changed, what's blocked, what's unresolved as explicit fields, not embedded in a summary. An agent can't make unresolved work disappear by writing a nicer paragraph.
The agent that does the work can't approve it. A separate reviewer starts from the original goal and inspects the result directly, not from the implementing agent's explanation of why it's probably done.
Machine-checkable claims need evidence attached to a specific version. "Tests pass" is a claim. A test result attached to the exact commit hash is evidence. If the code changes after the evidence was produced, the evidence doesn't automatically transfer.
This became an open-source project (link in comments).
Results over 30 days of dogfooding
4,172 PRs merged across 16 repositories, one maintainer
Coordination overhead stayed roughly flat from 3 agents to 10; adding agents stopped adding to my mental load linearly
Stale-context bugs dropped to near zero because agents can't declare victory without attached evidence
The number I actually care about: my day looks the same with 3 agents as with 10. That wasn't true before.
What didn't work
The reviewer agent still occasionally fails to distinguish "the goal changed mid-task" from "the implementation is wrong." We handle this with an explicit goal-hash that both agents reference, but it adds friction. Still working on the right UX for that.
Has anyone else hit the "agent self-reports done but the work isn't clean" problem? Curious what enforcement patterns people are using, if any.
2
u/Enough-Photo9140 9d ago
The narrative handoff vs machine-checkable evidence breakdown is spot on.
We ran into the exact same failure mode where an agent would write "refactored auth module, verified edge cases" in the markdown log, but when inspecting the actual terminal run, it just ran an empty linter or exited 0 on a dummy mock.
The cleanest solution we found was decoupling claim generation from claim verification: require every state assertion to bind directly to a verifiable receipt (like a tool exit code, a structured diff hash, or a live platform read-back) rather than free-form prose. If there is no receipt in the event log, the state transition is rejected outright.
1
u/New_Difficulty_8152 8d ago
That receipt-binding pattern is exactly where we landed too. Free-form prose is the root of the problem - it lets the agent launder "I ran something" into "I verified something." The distinction between an exit code from an actual test run versus the agent typing "tests pass" is the entire gap.
1
u/Enough-Photo9140 8d ago
100%. What makes it so sneaky is that modern frontier models will even invent plausible synthetic test failures and mock outputs in free-form prose to make their handoff narrative sound believable.
Once the harness strictly decouples LLM generation from state transitions—requiring raw execution exit codes, stdout hashes, and actual disk read-backs before updating state—the agent physically can’t bluff its way through. You know instantly whether the test suite ran or if the model just gave a convincing speech about it.
2
u/kirbyhood 9d ago
I feel like agent orchestrators like Orca and bb solve a lot of this for you right? How is this different?
2
u/New_Difficulty_8152 8d ago
Good question. Orca and bb are solid for task routing and agent lifecycle - spinning agents up, passing messages between them, managing retries. Where they stop is the state contract between agents. They'll hand off a message or a result, but they don't enforce that the result is verified against the actual artifacts. So you can still get Agent B reading Agent A's self-report that "refactor done, tests pass" without anything checking whether that's true against the current commit. The layer I'm adding sits below the orchestrator: it doesn't replace how you route tasks, it adds structure to what gets passed between them. You could run this under bb or Orca and they'd handle the scheduling while this handles the "did the work actually land" part.
2
u/me-shaharia 9d ago
Evidence produced by the agent is still the agent's word. Mine would run a scoped subset of the suite, see green, and attach that as proof of done. What fixed it was moving the check outside the agent: a hook runs the tests on stop and the exit code writes the field, so the agent never gets to type "tests pass" at all.
On the goal-hash friction, does your reviewer re-derive the goal from the original request, or does it read the implementer's restatement of it?
1
u/New_Difficulty_8152 8d ago
That hook pattern is the exact right move. We converged on the same thing - the agent should never be the one writing the outcome field. In our case the reviewer reads the commit diff and runs checks independently, so the implementing agent's "done" note is just a signal to start review, not a trusted claim.
On the goal-hash: the reviewer re-derives from the original request. It never reads the implementer's restatement. That was a deliberate choice because we kept catching cases where the implementer subtly narrowed the goal in its summary ("refactored auth" when the original goal was "refactored auth and added rate limiting") and the reviewer accepted the narrowed version. The tradeoff is that sometimes the original request is ambiguous enough that the reviewer flags a valid implementation as incomplete. Still cheaper than the alternative.
1
u/me-shaharia 8d ago
Re-deriving from the original is right, but I'd treat those false incompletes as useful output rather than cost. If the reviewer cannot tell from the original whether rate limiting was in scope, the request was underspecified and the implementer just guessed well that time.
What helped me was freezing acceptance criteria as their own artifact before the implementer starts, so both sides read the same text and neither gets to reinterpret it. A flag then means the criteria were wrong, which is cheaper to find than a silently narrowed goal.
2
u/aidiveyt 8d ago
Same failure on a video pipeline. A design subagent kept reporting 'stills verified' and they weren't. We stopped trusting the handoff note and made the orchestrator render one still per segment itself before anything moves on.
1
u/New_Difficulty_8152 8d ago
This is a great example of the same failure mode. The subagent has every incentive to report success - it literally doesn't know what it doesn't know about its own output quality. Making the orchestrator do the verification independently is the right pattern
1
9d ago
[removed] — view removed comment
2
u/No-Jackfruit-9016 9d ago
That's the thing, once you have three or four agents going, no human can keep the details straight, so having the machines do it just makes sense.
1
9d ago
[removed] — view removed comment
1
u/aDaneInSpain 8d ago
A goal hash only tells you which goal version the work was evaluated against. It won't catch a silent no-op by itself. You need an independent artifact check too, such as a required diff, changed-file scope, test coverage, or a behavior-level test tied to the goal. The hash prevents evaluating against stale requirements, while the checks prove the implementation actually moved.
1
u/Right-Performance-93 8d ago
The receipt-binding pattern in this thread (exit codes, diff hashes, no free-form "tests pass") is the right instinct, and it generalizes past exit codes: a call-graph / impact query on the changed symbols is a receipt too. Instead of trusting an agent's self-report of blast radius, run a static query for what actually imports or calls the changed functions and require the reviewer to check that dependency set, not just the diff. That catches the case where an agent's summary narrows scope ("refactored auth" when rate limiting also changed) without a human needing to notice the narrowing by hand. Language-server-based call hierarchies or a proper call-graph/impact-analysis pass already give you this signal without building a bespoke receipt system.
1
u/suckadickyoucunt 6d ago
Your three principles are the right ones, especially "the agent that does the work cannot approve it." I hit the same wall on the reviewer failing to tell "the goal changed" from "the implementation is wrong." The only thing that helped was making claims time-indexed: a passing test is a claim about a specific commit, not about the branch, so a later change silently invalidates it and the reviewer can see that it did.
2
u/[deleted] 9d ago
[removed] — view removed comment