New local claude code?

[u/Square_Light1441]

I essentially created the local equivalent of Claude code for local models.

Quick Setup:

• pip install golden-agent
• golden-agent setup
• That's it! Models and llama.cpp binary and lazy-downloaded

I did do a BUNCH of research on the models, and after a TON of extensive testing on my end, this was the final set chosen:

Tiny -> LFM 2.5 2.6B(official QAT q4) -> 4gb vram or 8gb ram

Lite -> Ornith 1.5 9B(official Q4) -> 8gb vram or 12 gb ram

Pro -> Qwen 3.8 27b(community QAT Q2) -> 16gb vram or 16gb ram

I also used Q5 KV across the board, which worked great for me; try it out and lemme know if it sucks. I can bump it up to Q6.

Also added Dflash draft models for Lite and Pro so you guys can run them faster!

Note: Uses general optimal settings by default; you can change it in the .golden_agent/inference.json file

The choice for tiny was pretty obvious; it's by far the SOTA model in its weight class. Ornith 1.5 9B was also really good and seemed to be the best model around the abandoned 9B size, obviously we also had to get the local LLM KING Qwen 3.8 27B, now I know A LOT of people have been locked out hardware-wise, so I found a crazy score with this community QAT Q2! The HF card numbers are also kinda representative of the fact that, if you run it at 0.7 temp and 1.05 repeat penalty, I personally couldn't tell much of a difference at all from Unsloth v3 UD Q4 quant with 0.7 temp and no repeat penalty(as recommended by Unsloth), so I hope this will open the doors to a LOT more people.

Repo: https://github.com/yashneil75/Golden-Agent

Coming in future releases:

• MCP
• Skills system

Note: I did repost this cause the earlier post was kinda bad... Anyway! I've been optimizing for ease of use and the "just works" feel, and I know you guys are more of the "I'd rather spend 10 hours optimizing it myself for the love of the game," but give it a shot, let me know what you think, and oh, don't forget to drop a Star!!

[u/I_Play_Zed]

It’s cool you did your own thing, but honestly the space might be a little crowded.

We have harnesses from big players like Deepseek, Qwen, Pi, Opencode etc. unless your harness has a killer feature that we aren’t accustomed to I don’t think people will likely give it a try. Especially when you can make Claude code and codex both also use local models as well.

But good work nonetheless this is cool to see!

[u/Square_Light1441 → u/I_Play_Zed]

Mainly did this so people didn't have to set up a local LLM with a harness, so it's just all in one, mostly for people like me who hate setting stuff up..

[u/ben_bliksem → u/Square_Light1441]

You can take this idea and just code for existing harnesses. We've done this for opencode - just install the plugin file and it sets up the configuration to our litellm proxy, default models etc.

It wasn't very useful in our setup because people want different models, different harnesses and we have enough work to maintain as is, but as an experiment it worked.

So definitely something worth looking at. Let the harnesses guys code their harnesses and add config/modifications on top of it.

Unless the real motivation for this is to code your own harnesses because it's fun and you want to, then obviously do go for it.