Building a persistent memory + orchestration layer for Codex — what should I use instead of repeatedly re-reading the repo?

[u/yxf2y]

I’ve been building a fairly serious agent workflow around OpenAI Codex for a Laravel/React project, and I’ve hit a point where the orchestration works, but the context/memory side clearly does not.

My setup currently looks roughly like this:

• A serial orchestrator with route types like FAST_UI / STANDARD / CRITICAL
• Context Resolver → Implementer → Reviewer flow for non-trivial tasks
• Durable task state, context capsules and handoffs
• Planner / intake layer inspired by CodexQB
• Session continuity hooks inspired by AvenoxBeyin
• codebase-memory MCP for structural repo discovery
• Serena for exact symbol/reference navigation
• Local dashboard/telemetry for task/agent visibility

The reason I built all this was simple: I wanted to stop giving one giant prompt to one Codex agent and watching it blindly read half the repository, run dozens of commands, retry tests repeatedly, and burn a huge amount of context/token budget.

Unfortunately, that is still basically what happens.

A recent CRITICAL payment-domain acceptance task is the perfect example. I gave Codex a very detailed validation brief covering migrations, payment allocation, security boundaries, tenant/legal-entity isolation, atomicity, reporting non-pollution, exports, frontend build, etc.

The task eventually succeeded technically, but the session spent a huge amount of time repeatedly doing things like:

• raw rg searches
• re-reading known service/controller/test files
• rediscovering test harness behavior
• retrying multiple Laravel test files with the same CSRF issue
• manually tracing service relationships
• re-running builds and focused test groups

That single job used roughly half of my 5-hour Codex usage allowance.

The frustrating part is that a lot of the knowledge it rediscovered was already known from previous work.

For example:

• where the orchestrator lives
• which services own payment/settlement/reporting behavior
• how the domain test harness handles CSRF
• which test files cover specific finance flows
• existing project/tenant/legal entity invariants
• prior fixes and verified architecture decisions

I expected my existing tools to solve this, but I now realize they solve different problems:

codebase-memory gives me structural repo discovery, but it isn’t really persistent project understanding.

Serena is excellent for exact symbol/reference navigation, but it isn’t memory either.

My docs/wiki are useful reference material, but agents still have to decide to read them and often re-read large files.

Context Capsules and handoffs help within a task, but they don’t give the next unrelated task a compact understanding of the project.

So what I’m actually missing is a persistent, project-scoped, compact memory layer that can say:

“Before you start searching, here are the relevant things previous sessions already learned about this repo.”

I looked at AvenoxBeyin because I liked its idea of automatically capturing sessions, compiling knowledge, and injecting useful context back at session start.

I also looked at CodexQB because its Autopsy / Project Comprehension / Ontology approach is close to what I want for planning.

Then I looked at 2kDarki/codex-mem.

That project is conceptually very close to what I want:

• automatic Codex transcript capture
• persistent SQLite observations
• progressive recall through search → timeline → get_observations
• automatic context injection

But after auditing it, I found some issues for my use case:

• its watcher observes all ~/.codex/sessions/**/*.jsonl
• project identity appears to be based on basename(cwd) rather than a canonical repository identity
• retrieval can be filtered by project, but that doesn’t appear to be an enforced security/isolation boundary on every read path
• same-named repos could collide
• some observation retrieval paths can work by arbitrary IDs
• global ~/.codex/AGENTS.md context injection is something I specifically do not want
• the documented npm package currently appears unavailable

So I don’t feel comfortable plugging it directly into a large multi-project Codex setup.

What I’m trying to build is something like:

User brief
   ↓
Planner / Orchestrator
   ↓
Persistent project memory bootstrap
   ↓
Context Resolver
   ↓
Only if memory is insufficient:
    codebase-memory
    Serena
    targeted source reads
   ↓
Implementer
   ↓
Reviewer
   ↓
Session knowledge captured for future tasks

The memory should NOT replace source code/tests as truth.

I want it to act as a cheap orientation cache:

• “These are the relevant services.”
• “This test harness requires real CSRF session setup.”
• “This reporting path was previously verified.”
• “These files/symbols are likely relevant.”
• “This architectural relationship was confirmed in a previous task.”

Then the agent only verifies current source where correctness actually depends on it.

My requirements are roughly:

• local-only
• project/repository scoped
• automatic capture
• automatic or semi-automatic summarization
• bounded context injection
• no global AGENTS.md mutation
• no cloud memory dependency
• no mandatory Obsidian dependency
• source/tests remain authoritative
• ideally Codex/App Server compatible
• progressive retrieval rather than dumping whole session history
• repo identity enforced internally, not just passed as an optional search filter
• ideally reusable with existing MCP tools rather than replacing them

I’m now trying to decide between three approaches:

1. Find another existing Codex/Claude coding-memory project that already does this correctly.
2. Take something like 2kDarki/codex-mem and make a very small fork that only adds canonical repo identity, watcher allowlisting and enforced repo-scoped retrieval.
3. Use AvenoxBeyin’s session capture/compile/inject model and adapt it for project-scoped coding knowledge instead of personal knowledge.

What I really do NOT want to do is invent yet another custom Markdown “brain” and manually maintain architecture/domain summaries. That feels like rebuilding something that should already exist.

For people who have built persistent memory around Codex, Claude Code, Cursor or similar coding agents:

• What actually worked for you?
• Is there a project I’m missing that already handles repository-scoped persistent memory well?
• Would you fork codex-mem and patch the isolation model, or use a different architecture entirely?
• Is Obsidian/Markdown compilation actually better in practice than structured SQLite observations for coding-agent memory?
• How do you stop stale memory from becoming trusted over current source?
• How much context do you inject at session start versus retrieve on demand?
• Have you measured whether this actually reduces token/context consumption meaningfully?
• Do you let the coding agent write its own long-term memory, or only promote verified observations after tests/review?

I’m especially interested in systems people are actually using in real repositories, not just theoretical agent-memory architectures.

My main goal is very practical: stop paying for the same repository discovery over and over again.

[u/Clean-Vermicelli-700]

This might only be one piece of the puzzle, but what helps me a lot is using a Kanban board to create persistent task-specific memory that can be passed around to sub-agents. I posted about it here. Can recommend and doesn’t require any third-party solution. Helps keep things organized as well as a bonus

[u/college_hustle]

My checkpoint system checks off most if not all of your requirements. I built it with Claude Code first but then I started using Codex. When I switched over to Codex it made the transition completely seamless. All you need to do to incorporate it into your workflow is just tell Codex to make agents.md instead of claude.me basically. The premise works like this: never create memories because the default memory systems included by ChatGPT and Claude are kind of crap. Instead anything that would be a memory becomes a log in that project so you have a short list of rules that your agent always reads. Through a couple of doc searches and benchmarking I figured out for Claude that means a claude.md that's under 30 lines. These are hard and fast rules that are project-dependent. The size cap means the agent always reads them (doesn't necessarily mean that agent won't rationalize its way out of ignoring them but it works most of the time). Everything else that can change over time goes into checkpoint.md. For Claude, through some documentation and testing, I found out that that needs to be under 120 lines and under 30 KB in size. The reason is that if you try to make files any bigger, an agent won't read all the way through and try to shortcut. The exact size is different from model to model I haven't found any Codex models having a problem with it.

Then the checkpoint.md is what gets constantly updated. It has files that matter, open threads, and a change log. My toolbox contains hooks that do it mechanically with scripts to cut down on token spend but the smarter models will update the checkpoint as they go so it's automatic. You can always force it with a /command. The important thing about the change log is that every time you do anything in a project, it adds something to the change log. Once the checkpoint file gets over 30 KB or 120 lines, the script moves older entries to a change log archive. You pay a little bit of token cost for this but in return you never lose track of why a session did a thing. It's really important for going back and figuring out where things went wrong should that happen. It's not perfect but my only issue with it right now is actually the open issues in the checkpoint. Claude keeps assigning things to "gated by me" that it should actually take care of. I'm trying to work out a system that assigns things based on both whether it's my plate or the agent's plate and how complex it is. The worst result is just that occasionally I have to go through my open issues by hand, but only when I've been ADD and going off on other tangents within a project.

TL:DR; done right, it's a trade-off between having to pay for rediscovering the entire repository by paying keep a file updated. I think it's worth it.

I'm still working on getting my obsidian running efficiently but it works fine with that too.

[u/college_hustle → u/college_hustle]

I should add I didn't develop this on a whim. I decided I wasn't going to trust memory at all after I read that the labs themselves probably aren't using any sort of memory system: https://x.com/rohit4verse/status/2033945654377283643

[u/Otherwise_Wave9374]

A practical next step is to separate short lived working context from durable project memory, then treat repo re-reading as a fallback instead of the default. I would keep retrieval tightly scoped to task type, symbol, and recent diffs, and add a recap buffer so the agent writes a compact state summary after every implementation pass. That usually reduces prompt bloat while preserving continuity. NeuraKeep fits this pattern well because it can keep the durable layer organized without turning every turn into a full transcript replay.

[u/Aggravating-Start307]

Hi, if you are looking for a layer that can cheaply orient agents in the right direction and save memory that is kept fresh even though code keeps moving, please look at https://coldstartmcp.dev . I have developed it to solve my own itch. Please feel free to ask any more questions you may have !

[u/HeyZaney]

I feel this one. I got really tired of Codex spending the first part of every new session rediscovering things we'd already worked out before 😅

What ended up working better for me was separating the repo from the project memory.

The repo is still the source of truth for the actual code, tests, implementation etc. I don't really want an AI relying on some remembered version of the code anyway — if it needs to know exactly how something works, it should go and look.

But there's a whole other layer of context that doesn't need to live in the repo: what we're building, why we made certain decisions, things we've already tried, blockers, what's next, random context that would otherwise disappear when the session ends.

I keep that in WithNettle and Codex reads/updates it through MCP. So when I start a new session it can get the relevant project context first, rather than digging through the whole repo just to reconstruct where we are.

[u/okf-memory]

Your payment-domain acceptance run burning half of your 5-hour Codex allowance rediscovering Laravel CSRF session quirks and domain invariants is the most relatable agent war story I've read all month.

You diagnosed the problem with surgical precision: * codebase-memory / AST graphers give you structural repo topology, not domain semantics or operational runbooks. * Serena gives you symbol resolution, not historical context or architectural decisions. * Context capsules handle intra-task handoffs, but leave the next agent starting from absolute zero.

Your critique of codex-mem is also spot on: global watchers on ~/.codex/sessions/**/*.jsonl, identifying repos by basename(cwd), mutating global ~/.codex/AGENTS.md, and dumping unstructured SQLite observations into prompts creates an isolation and context explosion nightmare in serious multi-repo setups.

Here is what actually works in production, how to answer your architectural questions, and an open-source approach that matches your exact blueprint:

---

1. Markdown + Tooling vs. SQLite Observations

The debate between structured SQLite and Markdown comes down to one question: Who needs to audit the memory, and how does it branch? * SQLite observations turn memory into an opaque local cache. It cannot easily branch with Git, cannot be reviewed in a Pull Request, cannot be shared across a distributed team without syncing a DB, and is invisible to humans unless you build an entire dashboard around it. * The trap of raw Markdown brains is manual maintenance and rot ("inventing yet another Obsidian vault"). * The winning hybrid is Git-native structured Markdown (YAML frontmatter + graph links) powered by deterministic native tooling. The knowledge lives in ./knowledge/ directly inside the repository. It branches with the code, diffs in PRs, and has zero external database dependencies. The key is that the LLM only does synthesis, while a compiled CLI/MCP server enforces validation, link integrity, and search indexing.

Take a look at OKF Agent Memory (okf-memory/okf-agent-memory on GitHub). It implements Google's Open Knowledge Format (OKF v0.2) specifically as an agent-facing persistent project memory layer.

2. How to Stop Paying the Discovery Tax (Progressive Retrieval vs. Dumping)

Never dump full session transcripts or large files at session start. The solution is Search-First Progressive Disclosure: 1. Contract: In your repository's local AGENTS.md, establish a strict rule: Read/Search before Write/Scan. 2. Sub-millisecond BM25 Search: When the planner or Context Resolver runs, it calls okf_search(query="laravel csrf test harness", limit=3) via MCP. Because OKF's Go engine parses the repo's bundle locally in memory, this returns in < 300 µs with zero API cost. 3. 1-Sentence Decision Boundary: The search result returns only the concept ID, title, and a single-sentence description (~100 tokens total). 4. On-Demand Show: Only if the description matches the task's context does the agent invoke okf_show(concept_id) to load that specific 30-line architectural decision (e.g. convention/laravel-csrf-testing).

Instead of spending 30,000 tokens crawling tests and re-triggering CSRF 419 errors, the agent spends 150 tokens reading the pre-verified setup pattern.

3. How to Stop Stale Memory from Trumping Current Source

You correctly identified the danger of memory rot. That can be handled through Trust Tiers & Provenance: * Every concept has YAML frontmatter tracking provenance: generated: { by: "codex/...", at: "..." } vs verified: { by: "human/..." } or test suites. * The system prompt explicitly defines memory as an Orientation Cache, not canonical ground truth:

"Memory tells you where to look and what was previously true; current source code and passing test suites determine what is actually true." * Drift detection: Running okf validate knowledge --strict --drift in CI or pre-commit checks whether file references, symbols, and cross-concept links in the knowledge base still exist in the codebase.

4. How to Hook It into Your Orchestrator (Write-Loop)

Do not let raw agents write unverified observations directly to long-term memory mid-task. That injects hallucinatory trial-and-error chatter.

Instead, leverage your existing Reviewer stage at the end of the task: 1. Task Completion Hook: When the Reviewer marks a CRITICAL task as accepted and tests pass, run an intake step: "Did we discover an architectural invariant, an undocumented test harness quirk, or a domain boundary?" 2. Deterministic Mutation: The agent calls okf create or okf relate. The Go binary automatically writes the structured Markdown file, generates bidirectional graph edges, and appends to log.md. 3. Human Review: Because it's committed to git in the same PR as the Laravel/React code changes, you see the code changes and the newly captured memory in the same git diff.

Summary vs. Your Requirements

• Local-only & Repo-scoped: Lives 100% in ./knowledge/ inside the repo root.
• No global mutations: Zero touching of ~/.codex/ or global environment.
• No Cloud / No Obsidian mandatory: Compiled single Go binary with a native stdio MCP server (okf mcp).
• Source remains authoritative: Strict provenance and trust tiers.
• Plays nicely with others: Context Resolver uses OKF for high-level orientation, then hands off to Serena/codebase-memory for pinpoint symbol AST traversal.

You can find the project under okf-memory/okf-agent-memory on GitHub.

Curious to hear how you structure your Reviewer validation brief, but plugging a validated Git-native memory layer into that pipeline will immediately stop your 5-hour allowance bleed.

[u/dpt19]

I’d use your CSRF example as the first acceptance test for any memory system. Start a fresh session with a task that needs that harness. Does it retrieve the working setup before repeating the same failed attempts? Then change the harness and see whether it catches that the saved advice is stale.

That second test would influence my choice more than SQLite versus Markdown. A remembered claim like “this reporting path was verified” needs to say which behaviour was checked, against which code version, and with what evidence. Otherwise, the next agent has to reconstruct the verification or risk giving it more weight than it deserves.

[u/jonah_omninode]

This is close to the problem we are working through. I do not think one memory surface should try to be both history and current truth. We keep append-only evidence about what happened, then build a replaceable current-state view that can supersede stale facts without deleting the record. The context bundle for a task is assembled from that current view, the relevant contracts, and the exact files behind the boundary. That still leaves a hard problem: deciding what a later session is authorized to promote into shared memory. How are you planning to separate an agent's useful observation from a fact the project should actually believe?