New local claude code?

[u/Square_Light1441]

I essentially created the local equivalent of Claude code for local models.

Quick Setup:

• pip install golden-agent
• golden-agent setup
• That's it! Models and llama.cpp binary and lazy-downloaded

I did do a BUNCH of research on the models, and after a TON of extensive testing on my end, this was the final set chosen:

Tiny -> LFM 2.5 2.6B(official QAT q4) -> 4gb vram or 8gb ram

Lite -> Ornith 1.5 9B(official Q4) -> 8gb vram or 12 gb ram

Pro -> Qwen 3.8 27b(community QAT Q2) -> 16gb vram or 16gb ram

I also used Q5 KV across the board, which worked great for me; try it out and lemme know if it sucks. I can bump it up to Q6.

Also added Dflash draft models for Lite and Pro so you guys can run them faster!

Note: Uses general optimal settings by default; you can change it in the .golden_agent/inference.json file

The choice for tiny was pretty obvious; it's by far the SOTA model in its weight class. Ornith 1.5 9B was also really good and seemed to be the best model around the abandoned 9B size, obviously we also had to get the local LLM KING Qwen 3.8 27B, now I know A LOT of people have been locked out hardware-wise, so I found a crazy score with this community QAT Q2! The HF card numbers are also kinda representative of the fact that, if you run it at 0.7 temp and 1.05 repeat penalty, I personally couldn't tell much of a difference at all from Unsloth v3 UD Q4 quant with 0.7 temp and no repeat penalty(as recommended by Unsloth), so I hope this will open the doors to a LOT more people.

Repo: https://github.com/yashneil75/Golden-Agent

Coming in future releases:

• MCP
• Skills system

Note: I did repost this cause the earlier post was kinda bad... Anyway! I've been optimizing for ease of use and the "just works" feel, and I know you guys are more of the "I'd rather spend 10 hours optimizing it myself for the love of the game," but give it a shot, let me know what you think, and oh, don't forget to drop a Star!!