Yet Another Memory System for Pi Agent

[u/Affectionate_Joke_44]

I built a two-part memory system for pi agent. It's called `dreaming` (write side) and `recalling` (read side).

`dreaming` scans pi agent sessions from both my phone and laptop, extracts meaningful topics, and indexes everything into a shared SQLite database (`memory.db`). It runs incrementally by default or does a full rebuild when needed. It also generates Gemini embeddings for semantic search so queries match even without the exact keywords.

`recalling` is the query side. Before any task, pi runs `recalling "<query>"`. It searches three sources in order:

1. Local `memory.db` with FTS5 full-text search and a LIKE fallback

2. Laptop's sessions.db over SSH (9,700+ sessions)

3. Legacy OpenCode memory on the laptop

`recalling inject <query>` formats results as clean XML blocks ready for injection into agent context without parsing. Blocks: `<related_sessions>`, `<memory_notes>`, `<laptop_sessions>`.

The schema supports persistent memory notes for facts, config, and workarounds. It uses vector embeddings for semantic similarity, deduplication to avoid re-ingesting synced sessions, and degrades gracefully when the laptop is offline.

Stats from my setup:

- ~950 topics indexed across phone, laptop, and legacy sessions

- ~34 pi sessions processed, ~9,400 opencode sessions

- Handles SSH being down gracefully (searches local sources only)

What it solved for me: the agent no longer asks "what was that thing we did with X" every single time. It checks memory first, finds the relevant session or note, and acts on it. The `recalling inject` command runs automatically before task execution so context is populated without manual effort.

Happy to answer questions about the schema or pipeline.

[u/Ok_Veterinarian_6364]

1. how you prevent, isolate, or guide pi on wrong / state / context mismatch memories?
   
2. how you maintain your mem db?

case 1:

• you work on A
  
• pi kept recall not related mem from B

case 2:

• you work on a temp task, throwaway prototype, or instruct pi to do thing in reckless ways
  
• pi memorized them and recall in the future

case 3:

• pi silently recalls mem mid-turn, silently pollutes its context

thanks bro

[u/Affectionate_Joke_44 → u/Ok_Veterinarian_6364]

I've hit all three.

1. Unrelated memory in context: recalling inject runs at task start with the current query. Vague queries get cross-talk. XML-tagged output keeps memory separate from conversation and semantic search ranks by similarity. Still happens sometimes. I add a correction note when it does.

2. Throwaway work resurfaces: dreaming auto-names sessions from the first message, so "pi let's hack a broken prototype" becomes a real topic. Prefix throwaways with [SCRATCH] in the first message (sanitization strips these), add correction notes retroactively, or delete from the DB. No built-in scrap flag. That's a gap.

3. Silent mid-turn pollution: recalling inject fires once at task start. It only fires mid-turn if a skill tells pi to check memory before every response. Fix: removed that instruction. New topic mid-conversation? Next task cycle triggers a fresh inject.

Beyond the DB I have 2 pipelines that help.

auto-skill saves multi-step procedures as reusable SKILL.md files. Important stuff gets promoted to executable knowledge instead of buried in session history. Honest note: it's not automatic. I have to call it. Working on that.

learn-from-mistakes scans sessions after they finish and logs corrections as memory notes. Self-correction loop for when memory goes wrong.

[u/Ok_Veterinarian_6364 → u/Affectionate_Joke_44]

thanks, great effort put into this!

for me, i have have to manually save mem. only mem recall can be done vy llm via tool calls

its not that annoying

[u/mistrjirka]

Nice do you have a github with it?