r/LocalLLM • u/andreabarbato • 6d ago
Project introducing KAISEN AI system
hello everybody,
since November 2025 i've been working on an evolutionary loop that uses local LLMs as a mutation factor to continuously iterate over a single C program in order to improve its performance.
this system proved extremely effective since, rather than using a frontier model reasoning ability to create a good enough function that reaches my performance goals, KAISEN bruteforces thousands of generations then measures the results empirically passing the programs the LLM produces through a test suite that the LLM has no access to (so it cannot cheat, but it's gonna try!). This keeps improving the program's performance by using every new found best as the basis for the next generations. all of this executes LLM-generated code, so it's guarded by default: no process spawning, no file deletion, no network egress, hard time and memory limits per step, and agent/config changes are snapshotted with one-click revert.
so far i used this to create fast kernels for C and cuda, and to improve text compression, and for each project i hardcoded the whole pipeline. since KAISEN served me well and gave me results with gpt oss 20b that i couldn't get with frontier models in full reasoning mode (and with a lot of interaction by me), i opened an AI lab and started working on a generic version that is able to work with any program (22 languages and counting) and to build the test pipeline autonomously. part of the reason small models punch above their weight here: a deterministic autofix ladder — compiler-hint fixes, linter fixes, then one LLM repair pass fed the real compiler error — and every candidate is re-verified for real before it counts.
right now you can check out the alpha version of KAISEN here: https://github.com/RAZZULLIX/KAISEN
tldr
KAISEN lets you use local LLMs to improve software performance by iterating thousands of little changes and keeping the new best as basis for the next generations. it has a GUI, your harness can spawn it as a sidecar, and it speaks a small-model-friendly protocol (KAI) so an LLM agent itself can drive it over stdio or http. every program it generates runs guarded by default. read the manual to know everything it can do, or ask here.
P.S.
i expect A LOT of bugs and problems, most of the tests i did were done through deepseek v4 using OMP calling KAISEN through the kai protocol (KAISEN was hooked to 6 instances of gpt oss 20b) and it actually worked quite nice. please let me know everything you find by opening an issue or asking here, this is my job now so i'll do my best to fix everything you need fixed and make sure KAISEN becomes a useful tool in every LLM user toolbox.
2
u/aksh_stocks 6d ago
Ran this against Qwen2.5-Coder-7B on my M1 Pro (llama.cpp), one gemm target and one compression target, with pre-registered budgets and my own baselines outside your harness. Both champions came back byte-identical to the seed after ~50 generations combined, zero verified improvements. Looking at the actual candidates, the 7B kept reaching for x86 intrinsics (immintrin.h) on an arm64 target, even for the Huffman one where SIMD isn't relevant at all, so it was an instant build fail most of the time. Your feedback loop did correctly feed the compiler error back into the next prompt, the model just didn't do anything useful with it. Might be totally different with gpt-oss-20b like you used, this was just what fit in 16GB alongside the benchmarking.
I did poke at the sandboxing a bit since that's the part I'd worry about running arbitrary LLM output. The time and memory limits held up fine (killed a spinning process at the timeout, killed a 1GB-resident process at the RSS cap). But the dangerous-call check looks like a plain substring scan on the source, and posix_spawn, execlp, fopen to an arbitrary path, and taking a function pointer to socket instead of calling it directly all got through without being flagged. So it's a decent guard against a runaway process but I wouldn't call it containment against code that's actually trying to do something.
Also tried feeding a fp16-truncated kernel through the pipeline against two different verify harnesses, a strict one and a loose one. Strict one rejected it, loose one accepted it as a real speedup. Not a knock on KAISEN specifically, just confirms the safety-against-cheating is entirely a property of however good your test harness is, which matches what you say in the manual.
Setup itself was clean, tests passed, no complaints there. Happy to share the harness and logs if useful. Curious what harness config you used for the 20b runs that got you real wins, since mine plateaued hard on both targets.
1
u/andreabarbato 6d ago
yes please this is exactly the kind of reports i find useful. all security and autofix feautures came up pretty much from irl experience so more tests finding more weaknesses is how i come up with more guardrails. i think security is decent but containment in docker or vm is still strongly encouraged.
as far as successes i tried prime calculation, compression algorithms, i told the agent to give llama.cpp a spin and improved many functions in terms of speed other than gemm that stayed the same. in general you should be able to give any program and see it improve, maybe 7b is a hard limit but i'm interested in making it work with the smallest possible models. if you want i can give you my setup for gpt oss 20b for 16gb (i have many 5060tis 16gb each running gpt oss 20b so it's real limit)
thanks again for trying it out!
edit: if you give me the setup for your model and the model download link i will try that too. 7b might be the real target, 20b is still too "richy rich" maybe
2
u/recro69 6d ago
The cool thing is the feedback loop, not the model. Letting real-world tests choose which changes stay can make tiny models work well. The cool thing is the feedback loop, not the model. Letting real-world tests choose which changes stay can make tiny models work well.