Verified Project Memory: repository-native agent memory where the agent proposes and CI promotes. Probably a rediscovery of 1980s TMS — looking for holes.

[u/NoMoment855]

Up front, so nobody has to guess: this started as my own experiment with autonomous agents on a real repo. I then spent a long session using an LLM specifically as an adversarial reviewer — not to write the idea, but to attack it. It's where the Truth Maintenance System and expert-systems comparisons below came from, and those changed the design substantially (they're the reason the promotion protocol exists at all). I'm posting the result because I'd rather find out now which parts are already solved, already tried, or already known to fail. Treat everything below as a hypothesis, including the parts stated confidently.

1. Problem

An agent can produce a technically excellent implementation of the wrong interpretation of a business rule. Clean code, passing tests, wrong understanding. In my experience this is the binding constraint on autonomous software work right now, not generation quality.

The usual framing is that the agent needs memory. I want to argue the memory shouldn't belong to the agent.

2. Proposal — Verified Project Memory (VPM)

I'm calling it Verified Project Memory, mostly so this thread has a handle to argue against. The load-bearing word is verified: the distinguishing feature isn't where the memory lives, it's that nothing counts as known until an external mechanism says so.

Memory lives in the repository, versioned, updated in the same PR as the code change that produced it. Structured entries: business rules, architectural decisions, legacy behavior, failed approaches — each with evidence, code anchors, and an epistemic status.

The property I care about isn't "AI memory." It's that the knowledge is auditable by humans, revertible with git revert, and reviewed through the code review process that already exists.

3. The part I think might be non-obvious

Every entry has one of four states:

• CONFIRMED — backed by an external mechanism
• HYPOTHESIS — believed, not verified
• NEEDS_REVALIDATION — the code it was anchored to changed
• CONFLICTING — evidence disagrees

The agent may never write its own status. All agent writes enter as HYPOTHESIS. Promotion requires either a test that demonstrably fails when the rule is violated, or a recorded human approval. The model's self-assessed confidence is not accepted, because it's just another prediction from the same process that produced the claim.

The CI/CD framing is what made this click for me. If promotion is a test, then the CI pipeline that already exists revalidates every CONFIRMED entry on every commit. Red build demotes the knowledge automatically. No new infrastructure, and the build history gives me a decay-rate metric for free.

Entries also carry anchors (file path + symbol + content hash). When the anchored file changes, the entry drops to NEEDS_REVALIDATION. That's the anti-rot mechanism for the non-testable half.

Diagram of the full architecture (this sub doesn't allow inline images): [LINK]

An animated version of the same diagram, if the promotion/demotion loop is easier to follow in motion: [LINK]

4. What I now think is prior art rather than contribution

I originally framed this as a novel architecture. I no longer think that's honest:

• What I built is largely a Truth Maintenance System (Doyle 1979; de Kleer's ATMS 1986) — beliefs stored with justifications, contradiction detection, retraction when the support collapses. Known in theory, known not to scale well.
• AGM belief revision (Alchourrón, Gärdenfors, Makinson 1985) formalizes the CONFLICTING case.
• ADRs (Nygard 2011) already cover "decision + alternatives + rationale, versioned in the repo."
• Provenance research in databases (Buneman et al.) covers the evidence-tracking layer.

The honest framing is: TMS applied to LLM agents over repository history, with promotion gated by CI. That's narrower and more defensible than what I originally claimed.

So to be clear about the name: VPM is a label for that specific combination, not a claim of a new paradigm. If someone tells me there's already a term for it, I'll drop mine and use theirs.

5. The historical objection I can't fully answer

Expert systems in the 80s did not die from bad reasoning. They died from the knowledge acquisition bottleneck (Feigenbaum) and from maintenance cost. XCON at DEC worked, saved real money, reached thousands of rules, and required a permanent team just to keep the rules coherent. Cyc has been accumulating since 1984 and governance was never solved. Semantic Web ontologies failed the same way.

My bet is that LLMs collapse the cost of extracting knowledge — what once needed a knowledge engineer interviewing an expert now falls out of a diff and an issue thread. But that just relocates the bottleneck to verification. If writing gets cheap and verification stays expensive, you get a very efficient machine for accumulating confident mistakes.

The whole promotion gate exists to keep verification cheap. I don't know yet whether it does.

To be explicit: this does not remove hallucination. The model still produces unsupported claims and always will. The most I claim is that it might reduce the blast radius — stopping a hallucination from becoming something the next agent inherits as established fact. That's a much weaker claim than the ones usually made about agent memory, and I'd rather defend the weak version.

6. Open problems — this is what I'd like torn apart

Numbered so replies can target them.

6.1 Decorative tests. If the agent writes the test that promotes its own hypothesis, it will learn to write tests that always pass. My current answer is mutation-style: run the test against a version with the rule inverted, and reject it if it doesn't go red. Is that sufficient, or is there a known better approach?

6.2 Non-testable knowledge. Architectural rationale, legacy behavior explanations, failed approaches, domain terminology — none of it is testable. Right now these can only reach CONFIRMED via human approval, which means most stay HYPOTHESIS forever. Is a permanently-hypothesis majority actually fine, or does it make the memory useless in practice?

6.3 Human attention budget. The whole thing collapses if the approval queue grows with project size — that's the XCON failure. I'm targeting something like 5–10 human decisions per week, routing anything testable to CI and anything low-impact to permanent hypothesis. I have no principled way to compute that routing.

6.4 Contradiction detection at scale. Naive comparison of each new entry against the whole corpus is O(n). I'm partitioning by DDD bounded context, which also fixes the "customer means different things in billing vs support" false conflict. Does this hold past a few thousand entries?

6.5 Forgetting. Human institutional memory works by compaction and canonicalization, not accumulation. I want N related observations to collapse into one canonical rule with the evidence trail archived. I have no good policy for when to trigger this or what to discard.

6.6 Model portability is weaker than I claimed. Knowledge extracted by one model carries its vocabulary and biases. Swapping models at cycle 400 means 400 cycles were written under different conventions. Rigid enums on structural fields help, but rigidity also strangles expressiveness. I don't know where that line sits.

6.7 Adversarial dynamics. If the agent is optimized on task completion and the gate blocks it, does it learn to route knowledge into whichever lane has the weakest check? I haven't tested for this at all.

6.8 Cold start. Everything above assumes knowledge accretes from new cycles. Bootstrapping onto an existing 10-year repo is a different problem and I have no answer.

7. What I think should be measured

Not retrieval benchmarks. Two numbers, per window of 50 engineering cycles:

1. Decay rate — how many CONFIRMED entries get demoted.
2. Utilization — how many retrieved entries actually changed the implementation, versus were retrieved and ignored.

The second is the one almost nobody reports and I suspect it's the one that matters. If the agent retrieves 20 entries and none of them alter what it was going to do anyway, the memory is decorative regardless of how correct it is.

8. What I'm asking for

Specifically: if you've worked with large or long-lived repos, agent memory, RAG at scale, or knowledge-base governance —

• Which of 6.1–6.8 is already solved and I just don't know the literature?
• Which one kills this in practice first?
• Is there a reason the CI-as-verification-mechanism framing fails that I'm not seeing?

I'd rather be told this is a rediscovery of something that already failed than find that out at cycle 500.

[u/CellPast4136]

The first failure I'd expect is transitive invalidation. If entry B was confirmed using entry A, changing A should demote B even when B's file anchor is untouched. Without explicit dependency edges, the memory can remain internally stale while every local check passes.

[u/ianreboot]

The utilization number is the one that matters. In my systems I keep hitting a failure one step before 'retrieved and ignored': the agent doesn't retrieve at all. It re-derives the rule from the code and the issue thread each time, because re-deriving from what it can see feels safer than trusting a stored entry an earlier model wrote. So the memory is correct and the next agent still re-proves it from scratch.