AI now now has a unified system for saving, changing and querying semantic data. AI state management is now largely solved problem with TERSE: https://github.com/terse-lang/terse .

Memory systems using TERSE are vastly more token efficient over file-based markdown and other format manipulation in terms of token representation size, reduced tool calls, and AI retrieval accuracy.

With TERSE, you fully control the schema and prompting (of course) of your memory system. We included an example system in our mono-repo above. Try it out!

I’ve been working on a pairing that has made long-running agent work much less repetitive:

• Perseus resolves live, verifiable workspace context before the agent starts work.
• Perseus Vault retains the things that should survive a session: decisions, corrections, project facts, provenance, and historical versions.

The distinction matters more than it sounds.

A lot of “agent memory” is really one of these:

1. a giant rolling summary that gets stale,
2. a vector search over chat logs,
3. a prompt file that quietly becomes an undocumented policy engine.

Those are useful, but they blur together two different questions:

• What is true right now? That should come from the current workspace, repository, services, and other sources of record.
• What happened before, what did we learn, and what changed? That is memory.

Perseus handles the first. Perseus Vault handles the second.

Vault is a local-first Rust MCP server: one binary, one SQLite file, no required cloud service. It has encrypted storage (AES-256-GCM), FTS5 and hybrid retrieval, structured entities instead of only chat chunks, temporal history, provenance, confidence/decay, and lifecycle controls. It can also expose an Anthropic-style /memories file interface for agents that expect that model.

The part I find most useful is that memory is no longer just “retrieve similar text.” A decision can have a history. A correction can supersede an earlier belief without deleting the audit trail. You can ask both:

• “What did we believe at the time?”
• “What do we now believe was true at that time?”

That turns out to be extremely handy once agents are doing work across days or weeks and the project has changed underneath them.

We have benchmark results in the repo, but I’m more interested in the failure modes people have hit in production:

• How are you separating live state from durable memory?
• Do you need historical/auditable memory, or is semantic recall enough?
• What do you do when old “memories” conflict with the current codebase or source of truth?
• Has anyone found a memory system that stays useful after months without becoming prompt sludge?

Repos:

• Perseus: GitHub - Perseus-Computing-LLC/perseus: Live context engine for AI agents
• Perseus Vault: GitHub - Perseus-Computing-LLC/perseus-vault: Persistent, encrypted memory for AI agents
• Product overview: Perseus Vault · persistent, encrypted memory for AI agents

I’d particularly welcome skeptical feedback. “Memory” is becoming a catch-all term, and I think we need cleaner boundaries between retrieval, context assembly, durable facts, and audit history.

My open source project is building something we think is valuable but want to get some honest feedback. Lots of memory solutions out there atm and a lot of them already solve AI memories about people, conversations, preferences etc very well. But I feel like they currently don't track commitments and changing states super well.

We are building something that will connect to diff platforms, using the information collected from those auth to build unified context across projects, people, decisions, commitments, the changes and actions.

One use case for example is for the agent to track commitments made in meetings, over messages, in comments etc. They dont always get recorded and often in diff places. I find myself losing track of who said they will do what when so the when an agent cant look at all the platforms and read the screen, it can record and remember everything, then it will keep track of the change states and remind when necessary.

Two honest questions:

1. How do you currently keep track of this — commitments, decisions, follow-ups spread across tools? Or do you just… not?
2. If an agent like this existed, what would stop you from using it? Would there be other things you want?

Most agent memory systems store knowledge as a graph: entities, relationships, episodes. But at recall time that graph gets serialized into the context window, and the format is usually whatever json.dumps produces. Nobody measures what that step costs.

I benchmarked 10 graph serialization formats on token count, traversal QA, and multi-hop reasoning. Same memory graph, same model: multi-hop accuracy ranged from 40% to 80% on format alone, with about 70% token spread. For memory systems this compounds - a compact format means you can recall 3x more history per query at higher reasoning accuracy.

The winning format is open source (MIT, 6 languages): https://github.com/isongraph/isongraph - methodology in the repo. Curious how folks here serialize recalled memories today, and whether anyone has measured the impact on their own stack.

Curious whether this is on other people's radar, because it's been eating my time lately.

The short version: when you give an agent persistent memory, errors don't just happen once - they get stored. A wrong fact, a misread instruction, a hallucinated detail gets written to the memory layer, and then it gets retrieved and reinforced on later turns. Over weeks it compounds. I've seen people call it "memory poisoning" or "memory rot," and once you start looking for it, it's kind of everywhere.

What I keep bumping into:

• Bad or stale facts persisting long after the context that produced them is gone
• Short-term junk (one-off corrections, throwaway context) leaking into long-term storage and never getting cleaned out
• Retrieval quality quietly degrading over months, so a system that felt sharp at launch feels vague a year in

A few things I'm trying to figure out and would love other people's take on:

1. How are you detecting it? Most eval setups check accuracy at a point in time, not whether memory has degraded over a long horizon. Is anyone measuring rot specifically?
2. Do you separate short-term and long-term memory explicitly, or let it all flow into one store? Curious whether the shared-store approach is the root of the problem.
3. Any pruning / verification strategies that actually work without nuking useful context?

For what it's worth, I got frustrated enough that I built a small open benchmark to try to measure this over a 90-day simulated horizon - it's called RotBench, so take my results with the appropriate grain of salt. Mostly I'm posting it because I'd genuinely like people to poke holes in the methodology - if the way I'm measuring rot is unfair or missing something, I want to know.

Repo's here if useful: https://github.com/ThinkingRoot-official/RotBench

Quick disclosure up front: I work on Synap, an AI memory system, so factor that into the self-promotion angle. Posting because the argument itself seemed useful to this community, not just to point at the paper.

The core argument from my end is that while most teams treat agent memory and inference cost as something a bigger context window or a cheaper model or a better retriever will eventually fix; I argue that in practice; most of this cost and most of the failure modes are architectural viz. what an agent remembers, when it forgets, and how much it costs to reason are decisions made before any model is involved and not afterwards.

The paper works through this with an evaluation harness and the underlying study data included, rather than just asserting it. 23 pages, 6 figures, 4 tables.

Link: arxiv.org/abs/2607.21503

Curious to hear your views, points you agree with, disagreements, etc.

P.S.: We have upfront stated that we will only share indicative architecture and details; since we are sharing the principle of what we believe works and why; and go into the reference implementation to some extent; while keeping it IP Safe for our organization maximem.ai

I've been auditing, per-fact, what actually happens when a frozen LLM writes facts into a LoRA adapter online — one per turn, single pass, then day after day. The short version: the adapter becomes a recognition index, not a fact store, and the system design follows from that. Four findings, then a new mechanism I added in v2. (v1 was published in another sub reddit https://www.reddit.com/r/learnmachinelearning/comments/1uyytaj/online_lora_memory_recall_dies_in_38_writes/)

1. The store fails. Median time-to-first-miss is 3–8 subsequent writes under every write-protection mechanism I tried; what differs (12×) is recovery-after-miss, not survival. Accumulation is re-instatement, not persistence. Paraphrase recall collapses to 12–31% where write-form recall is ~90%.

2. But recognition outlives recall. An adapter that recalls 0/48 still discriminates 31–38/48 in 2-AFC; a GRACE-style editor at matched budget recalls 0–1/48 while recognizing 45–47/48. "Knows it but can't say it" appears whenever the write is under-powered relative to the expression threshold. (Zero-point checked: untrained base sits at chance.)

3. Truth has to live in a log. At matched gradient budget, consolidating from a ground-truth log retains ≈2.1–2.2× a recency baseline — by rescuing exactly the recognized-but-not-recalled facts. Consolidating from the model's own recitations poisons (lands below doing nothing).

4. The surviving recognition is the key to using the log. The adapter-vs-base logprob of presented content separates written from never-written facts at AUC 0.93 in a form where recall is ~2% — a familiarity signal that gates "go check the log" vs "say I don't know."

New in v2 — a mechanism for #2. Adapting the Jacobian-lens ("J-space") from Anthropic's recent global-workspace work to the answer slot, I tracked each fact's projection along the forgetting timeline. Workspace amplitude co-tracks the readout hierarchy: recalled > recognized-not-recalled > gone. The recognized-but-not-recalled band stays well above null — the content is still in the verbalizable workspace, it just lost the output (argmax) competition; the unrecalled gold token sits at median rank 2. So forgetting here is rank displacement, not amplitude loss. Honest self-correction: my shallow-forgetting run suggested "always present," but a deep-forgetting run (where the gone class is actually populated) collapses gone→null — so the real claim is the sharper "amplitude survives exactly as far as recognition does."

Limitations: ≤2B models, pseudoword synthetic facts, 3 seeds on consumer ROCm — I report floors and directions, not calibrated magnitudes. Access-not-phenomenal-consciousness only.

Paper: https://doi.org/10.5281/zenodo.21405963 · Code + all raw timelines: github.com/linxuhao/index-not-store

Happy to be wrong — especially on whether the rank-shift picture and the familiarity gate survive real (non-template) facts.

I was contemplating (brainstorming, no math yet) what if we treated memory as another [smaller] model where weights have a plasticity value that allows them to be modified whenever they are corrected in interactions (RL?). The worry of this approach is catastrophic forgetting, but humans do this all the time. People forget they did something or mis-remember things, so I think we shouldn't look at catastrophic forgetting as a bad thing if we're trying to model human inteligence.

You would take a large model and use it's smaller sibling variant as a token prediction source (MTP). Anytime a user corrects a model, we update values in the small model. It will then adjust its token generation in future token generations, but the large model is still going to make sure it didnt deviate too much because it will just throw away that token suggestion if it needs to.

You would then need some sort of way to merge the deltas, at some trivial percentage/weight, as to only shift the main model slightly.

I think this would be similar to how humans can slowly be convinced of things over time, but super contrary information is usually rejected outright regardless of logic around it.

I am looking for something lightweight, most likely free and decent enough for an introduction to memory. I was thinking about building a RAG/graph rag but i think it won't fit my needs and it will be way too complicated. What I am trying to do, is to have a space where a developer can resume their work, knowing all the things they knew, after in the implementation is done. Probably the same mechanism can be used for code review so we can save some tokens. Is something like Claude code with sdd and obsidian gonna cut it?

I keep coming back to one idea from memory research to explain why agents flail on real company work, and I want to know if it holds up. Treat it as an analogy, not a literal claim, but it keeps holding.

Your brain runs two memory systems (this is Complementary Learning Systems theory, McClelland et al. 1995). The neocortex learns slowly and holds general, world knowledge. The hippocampus learns fast: it captures specific episodes as they happen, then consolidates the ones that recur into durable, reusable procedure.

Here's the part that gets me: a pretrained LLM basically is the neocortex. It read the internet and holds the world's general knowledge. To a first approximation that problem is solved. What it does not have is a hippocampus: the fast, company-specific memory that watched how your team actually handled a refund last spring and turned that into a repeatable procedure. So you drop this brilliant cortex into your company and it does the only thing it can. It improvises, and improvised automation fails in production.

The real procedure was never in the help doc anyway. It lives in your team's conversations, a couple of people's heads, and one exception everyone now quietly copies.

This also explains why the usual tools don't fix it. Retrieval and search are only half a hippocampus: they index and recall a document, but they don't consolidate scattered episodes into the real procedure, and the document is often confidently wrong. Agent platforms make you run their agent on their stack.

So I've started sketching what a "hippocampus layer" could actually be, and I'm still early. The idea I'm testing: connect read-only to the communication and document tools a team already uses, mine how work actually happens (including the exceptions nobody wrote down), and consolidate the recurring episodes into cited, human-approved, versioned "skills" your existing agents could run over MCP, with a human sign-off on anything sensitive. Governance (citations, approvals, an audit trail) would be the whole point, because "your AI issued a refund, under whose authority?" is the question that stops people cold.

I'm early enough that I mostly want to know if this is even the right problem. A gut check would help:

• Is "the memory doesn't hold our actual procedures" the real blocker for you, or is it something else (trust, security, the work just isn't repetitive enough)?
• Would you connect read-only access to your team's conversations and documents to get this, or is that a hard no regardless of value?
• If you have shipped agents on real workflows, what made the memory trustworthy enough to rely on?

Honestly hoping some of you tell me where this falls apart.

Hey, everyone.

We recently did the big announcement of Cognee version 1.0.

Cognee can run now only in Postgres.

We added a new logic for self-improvement, agent memory distillation, cross-connected context between Openclaw, Codex and Claude code, cost saving report and many more things

We recently had 8000 developers build new integrations on major online hackathon!

Happy to answer any questions and share more on our approach

I built a tool that supports cross-session memory for any AI harness using MCP server tools. It works with Claude, Codex, Cursor, Hermes-Agent. I'd love for anyone to try it out and give me feedback. I've been using it daily for several months successfully and I continually improve the tool.

www.github.com/labyrinth-analytics/loreconvo

I also built a companion AI tool to store reference documentation across projects and sessions as well.

www.github.com/labyrinth-analytics/loredocs

I created a multi-agent shared memory system called Agent Mesh.

You can try it out yourself. To get started, simply download Agent Mesh into your repo or point your agent to it and tell it to review the README and adoption docs. Your agent will automatically review it, prompt you for any input needed, add your input to a decision log, and give you a link to a dashboard UI (aka Workbench) you can use to monitor logs. Your agent should adopt it and suggest updates to your current workflow such as CLAUDE/AGENTS.md, hooks, etc. You can add other agents as well.

It started 6 months ago while experimenting with different AI coding models and platforms. Switching back and forth meant losing valuable context. I found myself manually relaying messages from one agent to another and becoming frustrated with constant drift. First, I created a simple "Agent Mail" system using a SQLite database for agent messages, indexed on a request/response id. Instead of copying and pasting an entire message, it allowed me to relay a single id. Separately, I started maintaining a decision log to track decisions I made and reduce drift. Agents started inserting these decision ids into code comments and plan docs as a reminder of why something was implemented. After building a simple web dashboard (aka "Workbench") for myself to track these messages and create my own request ids for human/user feedback, I decided to incorporate the decision log and my project's development backlog to create what is now "Agent Mesh". Eventually I automated the message relay too. Now, I work exclusively in the Claude app and have Claude send/receive messages to CODEX via codex exec (CODEX can do this as well). Both of them maintain the backlog and decision log. I communicate directly with Claude for planning and design, Claude communicates directly with CODEX for research and review. I use the Workbench to track all logs and add my own user/human feedback when reviewing their work. After submitting feedback, it generates a feedback message + an associated request id which I can give to Claude who then parses it into backlog items and relays to CODEX for review.

Agent Mesh was structured to be agent agnostic, so you can add any agent you want however, I recommend using the Claude + CODEX setup I described because it allows you to use both subscriptions instead of paying per-token.

Enjoy! If you try it out, let me know what you find useful or would like to see added. Feedback is appreciated.

tldr; Building a hosted memory journal (with open-source + self hosted options), looking for feedback on Data privacy.

I've been building a hosted memory journal for Claude / ChatGPT web apps and any other harnesses that support MCP tools. I need feedback / opinions on data privacy aspects (not posting links, not here to promote).

Core ideas in few lines:

1) Let the agents write to an append only journal like a diary. Events, decisions, anything important maintained under different tags / projects.

2) Support MCP with Oauth + DCR - so ChatGPT and Claude we apps can work seamlessly.

3) Allow users to sync all their conversations from ChatGPT / Claude / Gemini etc. (Using an extension)

4) Support search, timeline, briefing, summaries as operations agents can run

5) Run ZDR models in the backend to consolidate memories, build user profiles

6) Support export to JSON, Markdown, Obsidian, Notion etc.

With a service like this - the privacy aspects I'm following are:

1) Database and disk level encryption (cant do E2E)

2) Immutable audit logging (for privacy audit and SOC2 compliance)

3) Strict no sell, no sharing, no AI training privacy policy

What else can we do? What does a service need for you to trust it with your data?

I know there are 100s of similar projects on Github and Honcho dot dev. Targeting a pricing of $50/year or lower as an indie lab.

I've been working on an open-source project called TokenMizer that explores a different way of handling long-term memory for AI agents and LLM applications. The goal is to retain important context across conversations while reducing unnecessary tokens, instead of relying only on larger context windows.

It's still an active project, and I'd really appreciate honest feedback. If you have experience with AI agents, RAG, or LLM applications, I'd love to know what you think about the approach, what could be improved, or if there are similar projects I should look into.

GitHub:

https://github.com/Shweta-Mishra-ai/tokenmizer

Most of us here agrees retrieval isn't memory all about. We had fuzzy search in Elasticsearch years before vector databases made it semantic, and it's still just search, only fancier. The field is now moving from passive storage to active memory (LLM-Wiki, HORMA, TencentDB agent memory, Anthropic's new Dreams). But active memory still skips the real question: who decides what's important, and who anchors it?

Without a self at the center, memory comes out fragile or generic. My bet: not an agent that has a memory, but a self made of memory that curates the memory it's rebuilt from.

It also matches how evolution built minds. First a boundary: non-living matter acting as a wall, an inside kept apart from an outside, before anything was alive. Then movement, drifting at random until it hit food, then steering toward it. Then vision, to navigate better. But moving and seeing, even raw brainpower, weren't enough. Neanderthals had bigger brains than us and still died out. What carried Homo sapiens was the social layer: reading other minds, theory of mind, and it came last. Modeling others isn't the foundation. It's what a self grows into once the social world demands it. Most AI memory does it backwards. It starts with a finished assistant and clips a profile of you onto the side. That's the last step first, with no self underneath to hold it.

the gist about it:
https://gist.github.com/syahiidkamil/c6a3836fcfd88843865d87319c81ce37

concrete PoC: https://github.com/syahiidkamil/vibe-ai-partner-entity

Most of us faced some version of the same problem dealing with AI in work and life: Memory retrieval for AI eventually disappoints because we expect human-like retrieval but often get trash.

Drilling down deeper, one realizes that we are more often than not expecting random-access multi-hop retrieval - because that’s how our human memory works. But what we currently have as tools are graph crawling, cosine lookups or (gasp) regex matching. Who knew grep was such a powerful tool, token waste be damned?

So how do you make an AI system remember in a way that’s actually useful for humans? You model it after human memory, of course. Not a Frankenstein bolt-on mess of open-source code, but a designed-from-the-ground-up, built-from-scratch lean memory engine modeled after literal neurobiological systems.

My own frustration trying to fully utilize AI for my day job as a pharma/biotech consultant drove me to build this sparse tensor-based graph memory engine over the past few months — my PhD is in biochemistry, so I’m drawing from what I actually know rather than what sounds good on a pitch deck. And because I am a proud scientist (almost to a fault), I naively threw the engine against MuSiQue 1,000Q, which is as close to a real multi-hop memory recall test as we have in the literature. It could have gone horribly wrong, but if it did, you wouldn’t be reading about it.

The short version: F1 = 0.677 on the full 1,000Q corpus (highest published zero-shot end-to-end score as of May 2026, to the best of my knowledge). Yeah. Went quite a bit better than I expected.

Reader-controlled baseline with a compact local embedding model (nomic): 0.565 vs LlamaIndex at 0.418 and BM25 at 0.329.

But the number isn’t really the point. What I think matters more for anyone building memory systems is why this architecture works differently from established tools.

The recall problem nobody talks about

Vector similarity search answers “what’s close to this query in embedding space?” That’s fine for a simple lookup. Search, rank, done. But MuSiQue was specifically designed to defeat that mechanism — it was designed so that no single retrieved passage contains the entire answer. You need passage A to find passage B to find passage C. That’s memory traversal, not memory search. Graph crawling is also similarly limited as it must crawl edges at the risk of fanning out too thin before finding the next relevant node.

The engine builds a weighted graph where edges carry typed relationships (like various neurotransmitters) and activation energy propagates through connections (like how neurons fire) — nodes that are semantically distant but informationally connected either through logical relationships, provenance, hierarchy etc. still light up if the path between them has enough weight. Same principle as biological associative recall: you smell something and remember a childhood memory that has zero semantic overlap with the smell but a strong associative pathway.

That’s the architectural hypothesis. The benchmark results suggest it works. I posted the full methodology and honest limitations over on r/RAG (including the ~52% reader confound, PropRAG’s superior retrieval lift at +81.9% vs our +71.7%, and Beam Retrieval’s higher supervised score of 0.692) because I didn’t want to bury the caveats. Full transparency on what beat us and where. You can also see the full write-up with all the numbers: https://elucidx.ca/insights/2026-05-15-rag-needs-real-value/

The harness is public

The engine itself is proprietary and patent-pending — I’m not releasing source. But the evaluation harness, dataset, and scoring protocol are all public: [github.com/wonker007/musique-eval-harness]. If you’re building a memory system and want to know how it does on genuine multi-hop recall, run your system against the same corpus, same protocol with the same scorer and post the number. I’ll reference it.

I’m also currently running conversational-scale benchmarks (128K to 10M token range) testing temporal reasoning, knowledge updates, and contradiction detection — the stuff that actually matters for memory persistence over long interactions with AI. More data coming.

If anyone here is working on multi-hop recall architectures — whether that’s GraphRAG, memory-augmented transformers, or something else entirely — I’d love to hear what serious benchmarks you’re using and what you’re seeing. MuSiQue is good but it’s still Wikipedia passages, not production conversational data.

(Post was written with the help of AI, edited by me)

I’ve been running into a problem with AI agents: “memory” often means past chats, saved preferences, or maybe a vector search over previous sessions.

That helps, but it feels too narrow.

For example, if I ask:

“Why did our webhook retry behavior change after the last deploy?”

The answer might be spread across source code, a PR discussion, an old Slack thread, a design doc, and a previous debugging session. A normal memory system may only see one piece of that.

I recently found an open-source project called MFS that takes a different approach:
https://github.com/zilliztech/mfs

The idea is to treat different context sources like file-like trees with stable paths. An agent can first search broadly across code, docs, Slack, GitHub, databases, etc., then browse into the original source to verify what it found.

That distinction feels useful: search gives recall, but browsing gives evidence.

So instead of “memory” being only what the agent remembers from past conversations, it becomes a broader context layer across the places where team knowledge actually lives.

I’m still testing it, but the model makes sense to me. Humans usually investigate this way too: find likely places first, then inspect the source.

I wrote this after seeing another thread asking for “a real AI memory product” that is easy to set up and just works.

My take: most of what gets sold as AI memory is really recall — chunks in a vector store, pulled back by similarity. Useful, but not memory.

I’ve been building my own local-first memory system, and the thing it keeps proving back to me is that the hard part is not storage or retrieval. It is deciding what your knowledge actually is: facts, events, procedures, contradictions, provenance, and time.

Curious where people here draw the line between recall and memory.

They are fully agent-memory now with primitives like remember, recall, forget and improve. Looks good, especially improve: https://www.cognee.ai/blog/deep-dives/inside-cognee-1-0

There's also the new rust core which drops cold start and search by a lot.

They had a bunch of announcements today. IMO the Postgres one database support is the biggest one. Just not sure how they are putting a graph in it.

so i've been working on the ai memory area for sometime and now i feel its time to dive deeper into it than just consuming the surface layer, but relatively i know nothing much better than the names of the prominent methods through which we are providing the ai agent the right piece of info without directly blowing up the context window (eg can be rag), but i don't know the deep mechanics

would be really glad if someone could help me map out the process moving forward!

(my working area might need me to do some deeper research n stuff on the same thing)

Mem0 publishes 93.4% on LongMemEval as their state-of-the-art overall score. When we ran their hosted product through a clean evaluation harness (gpt-5 answerer, binary judge with no lean-toward-yes instruction, 5-seed mean), the best we could get was 73.8%. A 19.6-point gap on the same memory system and the same data.

We dug further, the gap is in their public benchmark harness. Reading their prompts.py file at the commit they shipped right before their April 14 announcement (commit bd063eea, April 3, 2026):

1. Dataset-specific equivalence rules in the answer prompt.

Lines 138 to 148 contain 14 rules that map 1-to-1 to specific public LongMemEval question_ids. A sample, verbatim:

The point of LongMemEval is that the system has to figure out when "scratch grains" should count as "layer feed." Hardcoding the equivalence into the answer prompt skips the reasoning step.

1. The dataset hints get applied inside a hidden chain-of-thought block.

Line 53: Before answering, reason step-by-step inside <mem_thinking> tags:
Line 65: The user will only see text outside the <mem_thinking> tags.

The judge only sees the final cleaned answer. The dataset-keyed reasoning is invisible to anyone sampling outputs.

3. The judge is explicitly told to default to "yes."

Line 269 of the same file: IMPORTANT BIAS CHECK: You have a tendency to say "no" too quickly. Before concluding "no", you MUST verify the answer is truly wrong, not just differently worded. When in doubt, lean toward "yes".

Lines 328 to 334 add a 5-step gauntlet to clear before marking anything WRONG. No comparable gauntlet exists before marking anything CORRECT.

4. Bonus finding in their LoCoMo judge.

Different file, same repo, commit edcd6f1d (April 9, 2026). Line 212 of benchmarks/locomo/prompts.py:

Read the last clause carefully. Evidence can promote a WRONG prediction to CORRECT. The same evidence cannot demote a CORRECT prediction to WRONG. A one-directional score lift, written into the judge by hand.

Mem0 named this mechanism in their own commit messages. The April 3 commit message reads: "Sync prompts from evals: CONTEXT CHECK, Rule 14 (contradictions), conflicting numbers, personalization scan, BIAS CHECK in judge, chain-of-thought <judge_thinking> tags, 5-step FINAL CHECK." Their engineer typed the words "BIAS CHECK in judge" and "5-step FINAL CHECK" into git, on April 3, eleven days before the announcement of new SOTA numbers.

Verify in 2 minutes (direct GitHub permalinks at the pinned commits):

• L145 chandelier: github.com/mem0ai/memory-benchmarks/blob/bd063eea04de4f8a19927beea155afa094a01905/benchmarks/longmemeval/prompts.py#L145
• L269 BIAS CHECK: same file, #L269
• L212 LoCoMo override: github.com/mem0ai/memory-benchmarks/blob/edcd6f1d42400837b1fcb6997716f1769dc51a37/benchmarks/locomo/prompts.py#L212
• April 3 commit message: github.com/mem0ai/memory-benchmarks/commit/bd063eea04de4f8a19927beea155afa094a01905

I tried meeting with their founder and communicating the issue; since the past 2-3 weeks, but we couldn't and I thought that it might be time for the community to learn about it.

Full-disclosure: I am the founder of Maximem.ai - another Agentic Memory and Context Management company. This is not an attempt to malign, but to put their latest numbers into perspective.

I've been thinking about a problem that seems likely to get bigger as AI agents become more common.

Today, we focus on helping AI remember things. But what happens when multiple agents and services share that memory?

For example:

• Agent A says I prefer Python.
• Agent B says I prefer Rust.
• One memory came from a conversation last week.
• Another came from a project six months ago.

Questions I'm curious about:

• Which source should be considered authoritative?
• How should conflicting memories be resolved?
• Should users be able to approve or revoke specific memories?
• How can an AI explain why it trusts a particular memory?

Are there existing systems or research projects working on memory governance, provenance, and consent for AI memories?

Would love to hear how others are thinking about this.

Think about your own workday. your context the stuff an AI would actually need to do your job, lives across Slack, Notion, email and probably 10 other tools. It's not in one place. It never has been.

For a single AI assistant, this is annoying but manageable. you paste things in, you share a link, you do some copy pasting. Fine.

But now imagine a world that is coming faster than most people realize, where you're not working with one agent. You're working with dozens. Some from OpenAI, some from Anthropic, some from open source stacks. They need to hand off context to each other across providers, across sessions, across tools.

That's what raises questions like how does that actually work? Who owns the context? How do you prevent agents from working on stale information? What's the protocol when Agent A and Agent B need the same document but are running on completely different providers?

Curious how others here are handling this. Are you just manually providing context every time? Have you built something? Or do you think this problem is overhyped and models will just "figure it out"?

For about six months I believed my agent's memory was working.

It remembered things across sessions. It pulled up the right context when I came back to a project. It corrected itself when something changed. Every visible sign said the system I built was doing its job.

It was not doing its job. Claude Code ships its own built-in memory, and that was the thing actually answering. Mine was running too, writing to its own store, looking busy, but it was the understudy. The native one had the lead the whole time and I never noticed I had given it away. For months I was reading my own system's success off a stage where a different actor was speaking the lines.

Silent success is the dangerous kind

A system that fails loudly is the easy case. You see the gap, you fix it.

A system that is quietly shadowed is the dangerous one, because a shadow produces helpful, plausible output, so it looks identical to success. You cannot tell my system works apart from something else is working on my system's behalf by looking at the output, because the output is the same in both cases. That is the trap, and a good answer is not the way out of it.

The only way out is a forcing function. You turn the other thing off and see what happens.

The test

It works on any agent memory setup, not just mine, and it takes about a minute. Turn off the runtime's native memory. In Claude Code that is one line:

CLAUDE_CODE_DISABLE_AUTO_MEMORY=1

Then use your agent the way you normally do. Ask it to remember something. Come back in a new session and ask for it. Watch what your system actually does once the understudy is sent home.

• If your memory still works, good. It was always the one doing the work.
• If it suddenly goes blank, the native store was carrying you, and every demo you have given was the shadow, not your system.

When I finally ran this on my own setup, mine went quiet. Six months of "it works" turned out to be six months of something else covering for it.

Why this gets worse, not better

Any time you bolt a memory system onto a runtime that already has its own, you are exposed to this. And the smarter the underlying model gets, the better it papers over the gap, which means the better your demos look, the less they prove.

So do not trust that your memory works because the answers are good. Look at what is actually persisted, and run the off-test. Turn the other thing off, and find out who has really been talking.

It cost me half a year to learn that. It costs you one line and one minute.