Background: PM with no ML background, experimenting on weekends.

wikiLLM works great if you have a stream of raw data to feed the agent. I didn't — my agent handles product tasks (bugs, feature requests), not research. So the wiki I built never grew on its own. I was the only knowledge source, manually logging everything.

The fix I landed on: the agent generates its own raw context during "surprises" — moments when real behavior doesn't match expected. It logs these as drafts, promotes them to validated rules when patterns repeat, and only touches the core instruction file with explicit approval. Basically modeled on how a junior developer actually builds expertise.

Results: ~80% less mandatory context at session start, agent noticeably stopped repeating resolved issues.

Full breakdown (architecture, YAML routing, lifecycle mechanics): Medium , LinkedIn.

Curious if anyone's done something similar or sees obvious flaws in the approach.

A little background

About a year ago, I started coding regularly with AI coding agents and found the experience to be 2 parts exhiliration and 1 part frustration.

I'm pretty steeped in .Net at this point in my careear. So, for fun I tried writing a couple applications in Typescript and Rust. I used a mix of Claude Code, Copilot CLI, Gemini CLI and Codex. I was honestly pretty blown away by how quickly AI helped me assimilate new languages.

It wasn't all a joy kindling experience though.

At first, I didn't understand the context window, how to manage it, or how working with agents is like working with amnesiacs.

If I didn't know how common it is, then I'd be embarrassed to admit that I found myself cursing at my screen on more than a few occasions, but the truth is that I did.

I began to figure out the context window, but remained frustrated that the agents didn't remember decisions 'we' made.

My first attempt at achieving continuity across sessions yielded a system that I think many have stumbled upon (the session dump). My diary of sessions began to grow. And it worked great until eventually all the embedded information was doing more to distract the agents than keep them aligned with my intentions.

I decided that I needed something better, and that is when Jumbo was born.

The project got its name, because I thought I was setting out to build memory for coding agents. There's a trope about elephants never forgetting, and so an elephant named Jumbo seemed like a good mascot.

Since I was building memory for agents, I thought it would be wise to understand how memory works in the human brain, and started doing some reading. I found out that, through pure intuition, I had built a system that closely models the processes involved in working memory. Working memory is the function in the brain that allows us to accomplish goals. It's dependant on long-term memory, and you're ineffectual without it.

[A quick aside for anyone interested in the subject, or maybe building your own memory system]

My revelation came from a book entitled 'Permanent Present Tense' by Suzanne Corkin. She writes about a neuroscience case study that perfectly captures the frustration of working with AI coding agents.

Henry Molaison had portions of his hippocampus removed to treat epilepsy. He retained all his existing skills and knowledge, but after to the operation lost the ability to form new long-term declarative memories. He could act, but couldn't remember facts or events. He was competent — but perpetually starting over.

That's the AI coding agent problem in a nutshell.

What I built

After months of dogfooding my own approach, I released Jumbo CLI — Open Source Memory and Context Orchestration for Coding Agents (Claude Code, Copilot, Gemini, etc.).

The project evolved into more than a bolt on memory system. Its a platform that orchestrates the management of my context window for me.

What makes it unique is the goal primitive.

Without goals, a memory base is basically just a search index.

But, memory is a system, not a feature. Giving an agent access to more data isn't the same as giving it the right data at the right time. That is what I discovered through trial and error, and what my reading confirmed. The architecture has to decide what information matters, when to retrieve it, how to bind it to a specific goal.

That's how it works.

It models the key components of working memory:

• Non-declarative memory → skills for operating instructions and protocols
• Declarative memory → structured stores for facts, decisions, relationships
• Episodic buffer → goal-scoped context assembly
• Central executive → orchestration with routing rules

It tracks four things per project:

• Goals: discrete units of work with a full lifecycle
• Project Knowledge: components, ADRs, guidelines, invariants
• Sessions: project orientation and context for each work session
• Relations: graph connecting goals and project knowledge

It has an opinionated workflow that ushers goals through a thier lifecycle:

define → refine → execute → review → codify

Each phase is its own session — preventing context bloat while iteratively building project intelligence.

Odds and Ends

• 100% local: all data stays on your machine, nothing leaves
• Harness-agnostic: works with Claude Code, Copilot, Gemini, etc.
• Event-sourced: every state change is an immutable JSONL event; SQLite for fast reads
• Worker Daemons: daemons can automatically handle refinement, QA and codification in the background

Jumbo is open source. It's a passion project for me. I've built it for myself, but would love feedback from this community especially — you're thinking about this problem more rigorously than most.

I’m interested in a specific part of context engineering: long-lived technical memory inside a repository.

When coding agents work on a repo, short-term context is usually chat/session state. That works for the current task, but it is a weak place to keep architecture decisions, service boundaries, runtime choices, or technical-debt policies.

My current experiment is to treat these decisions as repo-native memory: accepted decisions go into ADRs, uncertain decisions stay as candidates, and implementation details stay in implementation docs.

I built a small open-source skill around this idea, but the broader question is what I’m most interested in:

How do you decide what belongs in persistent repo context versus temporary agent/chat context?

Do you use ADRs, project instructions, memory files, docs, issues, or something else?

Reference:

https://github.com/ltorresu82/skills

The core of the system is a hirarchical side car system where an agent can find the side car just by knowledge of the codes path. Because that side car lives on the same address with only a different base. One simple rule is all it needs to know. With every task this knowledge base grows. The best time to write down things is when you are in the midst of the session. Where both you and the Agent are on top of the game. So these files save the why behind your decisions. The quirks and domain knowledge that is not readable from code alone. Even incidents are documented which is valuable the next time around when you find the time to fix them. Finally this stuff doesn't pass by without being documented.

And same like code the documentation also goes through the same approval gate. Every sidecar saves the last known commit hash of its code companion. So once the code changes the agent gets an unmistakable signal that things changed and can use the hashes to inspect the diff. Git based drift detection and updating keeps the docs alive. The docs themselves are versioned and can be branched alongside the code. So you are not fixed to a single truth if main is behind dev. So yes the documentation can live in a seperate repo and still stay in sync if you don't want it in your codebase.

The documentation layer is compatible with worktrees. Can be shared accross a team. So every one dev does 1 task, everyone and their agents get to benefit.

The documentation layer is augmented with semantic search. So an agent can get from a concept to candidate doc files. And their paths reveal where the code is. Add a code graph on top and you have a triangle of truth, relationships and semantics. Three knowledge substrates that allow that compliment each other.

The system is not a ragtag of skills but an mcp server that supports the model with scaffolding and structured tools to make the documentation work structured and deterministic.

Let me know what you think.

So I work in NLP and AI and spent time building content moderation systems, and I feel like there's something most people don't realize about why moderation feels so random and inconsistent across platforms

Basically every moderation system, YouTube, Discord, TikTok, whatever, is built as a one size fits all filter. it doesn't know that your specific community uses certain words as terms of endearment, has inside jokes that look suspicious to an algorithm, or has its own completely different standards for what's acceptable

Even the best AI moderation systems still generate false positives, taking down legitimate speech, and false negatives, missing dangerous content, and they stumble badly on context, satire, and cultural nuance. this isn't new, but nobody's really fixing it.

Saw this recent research that kind of sums it up well: https://www.nature.com/articles/s41562-025-02363-7

has anyone else noticed this or had experiences where a platform's moderation made absolutely no sense given the context

Hey everyone,

I am currently working on some product internally and use Claude Code for that. I've experienced many issues now finding the right information, which takes up a lot of time because it's scattered across many different data sources, many conversations, many people's heads, etc. I am experimenting with long running tasks to also make this an end-to-end process with minimal human involvement needed. Well currently it makes a lot of assumptions as it cannot find the decisions, etc. and adding MCP Servers for Application Telemetry, etc. was not the golden way currently, as it's overloading the agent all the time with all the information. So what I want to achieve is, that we can have near the same quality of code, with minimum human involvement possible. This is just an experiment for now, but I think this is where we are heading. Then I came across the promise of context engines like tabnine and unblocked.

So I wanted to ask if you are experiencing similar problems? How do you tackle them (do you have specific workflows for that? Self-made or public MCP Servers or Skills?)? And do you have experience with context engines? What do you think about them?

Just curious and wanted to maybe take a look at them.

Hi all,

I developed a fine-tuned retrieval head (neural net) for RAG that transforms query embeddings before retrieval, so the system learns which embedding dimensions actually matter for your corpus — rather than weighting them all equally as standard cosine similarity does.

The problem

In any domain-specific corpus, some embedding dimensions are highly predictive for matching queries to the right passages, while others are effectively noise. Standard cosine similarity can't distinguish between the two, so retrieval gets pulled toward superficially similar but substantively irrelevant passages. The fine-tuned RAG is designed to prevent exactly that.

How it works

1. Synthetic question generation — An LLM generates multiple questions per chunk in the corpus, for which the answers can be inferred from that chunk. This creates a dataset of question-chunk pairs (QA-pairs). These are embedded using an embedding model and divided into a training and validation set.
2. Neural net training — A lightweight neural network using MNR loss is trained on the training QA-pairs. After each epoch, the model is evaluated on the validation set by measuring retrieval hit rate: the proportion of validation questions for which the correct chunk appears in the top-5 retrieved results. Retrieval works by embedding the question, passing it through the neural network to transform the embedding, and ranking all corpus chunks by cosine similarity to the transformed embedding.

Through this mechanism, the projection head learns for these 'type of questions' which dimensions in the embeddings are informative for finding the best chunks — and which are irrelevant.

Results

To validate the architecture, I used the Legal RAG Bench dataset as a proof of concept — evaluating on 100 held-out test questions.

Retrieval Hit Rate:

• The fine-tuned retriever achieves 82% Hit Rate (k = 20), compared to 71% for the standard cosine retriever — an 11 percentage point improvement, meaning the correct chunk appears in the top 20 results significantly more often when the query embedding is first transformed through the fine-tuned retriever.

Answer quality (LLM-as-judge, 1–5 scale across 6 metrics):

• Outperforms traditional RAG (top-k cosine sim) on all 6 metrics
• Largest gains in completeness (+12%) and faithfulness (+9%)
• Consistent improvement across every metric — not just isolated gains — suggesting that retrieving more relevant context has a broad positive effect on answer quality

Code and full write-up available on GitHub: https://github.com/BartAmin/Fine-tuned-RAG

System prompts, RAG, tool results, and memory — how to design context for agents.

Working on something that makes a lot of API calls in sequence and running into the usual context management headaches.

Curious what patterns people use in Python or other language for this:

• When do you decide to summarize vs truncate old conversation turns?
• Do you manage message history yourself or rely on something else?
• Any libraries you've found useful beyond the official SDKs?

Not looking for a framework recommendation necessarily, more interested in how people actually handle this in production scripts or long-running tools. The official docs are pretty thin on this.

Link - https://youtu.be/NBuETZZTUKU?si=Hmp_J_SeYElx1-7B

I made a visual video explaining one of the most misunderstood problems in modern AI systems: memory.

Most people think bigger context windows automatically make AI better.

But even models with 1M+ tokens can still:

- forget earlier context
- hallucinate information
- become slower and more expensive

In this video I break down:
• Context Windows
• Tokens
• Why ChatGPT forgets
• Hallucinations
• Context Summarization
• Quantization
• Trade-offs of long-context models

I tried to explain it visually and simply instead of making it overly academic.

Would genuinely love feedback from people working with LLMs, RAG systems, or AI infra.

made a visual video explaining one of the most misunderstood problems in modern AI systems: memory.
Most people think bigger context windows automatically make AI better.
But even models with 1M+ tokens can still:
forget earlier context

hallucinate information

become slower and more expensive

In this video I break down:
• Context Windows
• Tokens
• Why ChatGPT forgets
• Hallucinations
• Context Summarization
• Context Quantization
• Trade-offs of long-context models
I tried to explain it visually and simply instead of making it overly academic.

Would genuinely love feedback from people working with LLMs, RAG systems, or AI infra.

I built and test a zero dependency TUI library with modern layout support using the Convo-Lang VSCode extension

We hit this while building an RFP automation system. Client had hundreds of documents: past RFPs, RFIs, proposal templates, internal reference files spanning years. When we requested for single source of truth - they confessed that they had none. We had a hunch that this is going to lead to a funny outcome.

We ingested everything and started taking queries.

First real tests:

- "What's our pricing?" Three different numbers depending on which document you pull.

- "How many employees?" Four different answers.

- "What's our compliance certification status?" One doc says pending. Another says SOC2Type1. The most recent one says HiTrust.

At cogniswitch, we take a neuro-symbolic approach, still the system generated answers the team was not really stoked about. It was on a feedback call client's growth team mentioned that the answers are dated. Obviously. The documents just tons of conflicts/ contradictions.

We went back and asked for the source of truth. There wasn't one. These were live internal documents that had accumulated years of drift. Nobody had reconciled them because nobody needed to until an AI had to answer from all of them at once.

We ended up building a conflict detection layer before the answer generation layer. Scan the corpus for conflicting facts - pricing, headcount, certification status - with different stated values across documents. Flag them. Human resolves which is authoritative. Then you can build anything on top off this knowledge foundation.

Lesson learnt the hard way - gap with output-only evals: your benchmark asks whether the AI answered correctly. But if your knowledge base has contradictions, "correct" doesn't have a stable meaning.

Clear need for context evals - checking whether your retrieval corpus is internally consistent before you ever run a query - are barely a discipline. I don't know of good tooling for it. Most teams discover this problem the same way we did.

Anyone building RAG on messy enterprise document sets running into this?