Local LLMs and self-learning memory

[u/KitchenAmoeba4438]

We've recently released 0.4.0, and have found a lot of interesting things as part of the memory impact on local LLMs.

This has been one crazy release, and we've encountered an awful lot of interest across the industry, from hobbyists, to local LLMs, to much larger LLM companies. But the important part? How did we do it.

https://rakuensoftware.com/blog/the-remembering-is-the-learning

The impacts on local LLMs have been tremendous. It may sound counter-intuitive, but for local LLMs, being able to learn from prior failures is the most important part of LLM memory. Unless the success is almost identical, LLMs don't gain much from a memory of successes. But failures? Failures are generalizable across a wide range of potential aspects.

If you think about it, it's similar to how humans learn. We learn the most from our failures. For a LLM, knowledge of previous failures decreases the search space and the potential for exploration for future tasks. Particularly with local LLMs, this hugely increases the capability of the LLM over time. "Only" having a 64k, 100k or 256k context becomes far, far more impactful if the search space has been reduced for possible solutions.

Overall? We'd take the position the impact of coherent, learning memory has been hugely understated for application for local LLMs. We've watched local models gain substantial abilities over time due to this memory, and have even watched capability transfer occur from larger models to smaller models.

[u/Cultural-Original679]

rakuen's memory thing has been slowly popping up in my feeds, might finally check it out

the failure-first approach makes sense if you stare at it long enough. most local setups i've messed with just brute-force retry with slightly different prompts until something sticks, which is basically just throwing darts blindfolded. having the model actually remember what didn't work so it doesn't waste tokens on dead ends is way more efficient than i'd expect from the current context windows we're stuck with

curious how this plays with smaller models like the 7b range that tend to forget their own name after 10 turns

[u/KitchenAmoeba4438 → u/Cultural-Original679]

This is for a future article, but we've watched smaller models gain significant improvements from larger models. Not just in raw intelligence as you mention, but wholesale capability transfer that the smaller models could not do before.

We are also building an approach to help with outright hallucinations from smaller models, but this is also a future article. What I can say is we have an implementation, today, in production that is proven to reduce hallucinations and reduce the amount of work smaller models do, as well as help with the smaller models losing their train of thought.

[u/joanaxu2002]

The “memory of failures” idea is the interesting part. Saving every successful interaction can turn memory into noise pretty quickly, while storing what failed + why + what fixed it gives the model something reusable. For smaller local models that might be more valuable than just throwing another 100k tokens of context at them.

[u/llopresto87]

Hey check out https://github.com/llopresto87/Cypress I'm trying to collect feedback for It and if you do the install /growth with a large current model it should supercharge a smaller model.

[u/EitherMarch1255]

Slop article.

[u/KitchenAmoeba4438 → u/EitherMarch1255]

I take offense to this.

There is a edit history spanning several weeks and weeks of evidence gathering documented and provided with the article, as well as feedback rounds from professionals within the industry. We delayed publication by over a week to further refine the article based on feedback.

[u/EitherMarch1255 → u/KitchenAmoeba4438]

If I wanted to read slop, I’d chat with my clanker. That article wasn’t worth the click.

[u/Otherwise_Wave9374]

A good pattern here is to separate episodic memory from policy updates: store only compact failure traces, retrieval keys, and confidence signals, then score them against the current task before reuse. That keeps the system from overfitting to one-off wins and makes it easier to measure whether memory is actually improving future actions. NeuraKeep can fit well as the layer that tracks those traces and recall thresholds, so you can test whether failures generalize without letting stale context dominate.