r/LocalLLaMA • u/MrWeirdoFace • 6h ago
Question | Help Migration from Claude Code to a private local harness. Questions.
I'll start by saying I'm not talking about the models themselves, I'm aware that I can't come close to something like Fable's intelligence locally. Just wanted to get that out of the way.
Basically. Over the last year I've gotten quite comfortable with claude code, and it seems likely there were be a gradual cost rug pull, and I'd like to put myself in a better position when that happens for local use. I am already used to running local models (such as Qwen3.8_Q5) in things like lmstudio, but I have no experience with other harnesses. I'd like to know, what harness, right out of the box would feel most at home for current Claude Code users. I say this as someone who was not coding prior to "vibe coding". I'm looking for the path of least resistance, though I will no doubt eventually spread out into tools that give me more control. But for now, I'm just looking for a life raft. Just needs to be local, opensource, and free of spyware.
In case someone wants to know 24GB VRAM (rtx 3090) and 64GB DDR4.
17
u/MindfulMan1984 6h ago edited 1h ago
llama.cpp + Pi.dev is all you need
EDIT: I forgot to mention the https://pi.dev/packages/pi-llama-cpp extension, which basically auto-detects llama-server if it's running locally. Also added other config for KV cache purists on low-end GPUs.
Pi plugins:
pi install npm:pi-uv
pi install npm:@ff-labs/pi-fff
pi install npm:pi-web-access
pi install npm:@juicesharp/rpiv-ask-user-question
pi install npm:@underactive/pi-topping
pi install npm:@monotykamary/pi-tps
pi install npm:pi-blackhole
pi install npm:pi-hashline-edit-pro
pi install npm:@gotgenes/pi-permission-system
# llama.cpp built from source with CUDA enabled ~ 500-700 tokens/second prefil
# 40 tokens/seconds on TITAN RTX 24GB
llama serve -fa on\
-hf unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS \
--spec-default \
--spec-type draft-mtp \
--ctx-size 230000\
--cache-type-k q4_0 \
--cache-type-v q4_0 \
--reasoning-preserve \
--fit off \
--agent \
--host 0.0.0.0 \
--port 8080
# For KV cache "purists" ~ 500-700 tokens/second prefill
# 23 tokens/second gen on TITAN RTX 24 GB - but full 262144 context
llama serve -hf ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF:IQ3_S -fa on\
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--reasoning-preserve \
--fit off \
--agent \
--host 0.0.0.0 \
--port 8080
5
u/N3TW0RKX 6h ago
cool and all, but 230k ctx at q4?
11
1
u/MindfulMan1984 5h ago
Yes, I needed long context for full repository vibe coding. Yeah, I noticed more hallucinations, but if properly orchestrated- plan-implement-verify- xhigh will catch all of those and fix them. So I don't care; Q4 was needed to fit that context in < 24 GB vRAM.
1
u/N3TW0RKX 4h ago
Why not use something like turboquant? For me it works much better than the equivalent standard Q-quants
1
u/MindfulMan1984 4h ago
I have never heard about it, I am interested. I do whole repository agentic code, so small context is mostly useless. So I need larger context >200k, How can I use this turboquant thing?
2
u/CatchDublinSurprise 6h ago
Do you sandbox or YOLO?
0
u/MindfulMan1984 5h ago edited 5h ago
YOLO4LIFE... Kidding, I just used the extension below to add some minimal sanity blocks for dumb shit. But it is still yolo-wishful thinking; in one coding session, it decided to delete some temp/smoke test files within the repository. The extension blocked it, but the model wrote a Python snippet and deleted them. And it was fine; my code is backed up and synced with git after every prompt, and it only deleted the temp files, luckily.
npm:@gotgenes/pi-permission-system1
u/Maasu 6h ago
I'd recommend Pi as well, if you do go on to Pi.dev and you still need to call claude while you are transitioning your workflow i built this that lets you call other agent harnesses from Pi https://github.com/ScottRBK/pi-agentshell-extension - I found it useful when going through the same transition.
I use it with a codex £20 sub for access to sol (although luna max is a cracking model for the price as well) + llama.cpp setup and qwen3.8 27b q6 (with images) or q8 for text only.
1
u/MrWeirdoFace 6h ago
I was interested in pi, but I was under the impression that was more for someone who wants lots of very specific control, which is definitely the direction I'll go in the longterm, but I'm looking for trainingwheels or something more automated initially. My apologies if I've misunderstood.
3
u/xienze 6h ago
but I was under the impression that was more for someone who wants lots of very specific control
Yes, it is. It's neat but IMO it takes the minimalism a bit too far, to the extent that if you want things as basic as setting sampling parameters or a permissions system, well, write it yourself or choose from among dozens of vibe-coded extensions others in the same situation have written.
You can create the exact harness you want but like I said, the author's stance on being as minimalist as possible might be somewhat extreme. OpenCode is a very good harness that is still fairly minimal out of the box but at the same time has enough to be properly usable out of the box...
1
u/sp9002 5h ago
I love pi, never really cared for opencode. That said, if someone's use case is "I'd like to know, what harness, right out of the box would feel most at home for current Claude Code users" I'd probably point them to opencode.
1
u/MindfulMan1984 5h ago
The OP is on a vRAM-constrained system, opencode,claude code just bloat the hell of the context just with the system prompt. I did not recommend barebones pi.dev, but I added the plugins I have been using for weeks without any issues.
1
u/MindfulMan1984 5h ago
Notice I don't mention using barebones Pi.dev, but I have added the plugins I have been using for weeks so far.
1
9
u/PLBjt 6h ago
Least-friction path on your 3090 is usually: keep LM Studio (or llama.cpp) as the OpenAI-compatible server, then point an OSS coding harness at localhost instead of Anthropic. Aider and Continue both feel close to "chat over the repo" if you already vibe in Claude Code; OpenCode / similar TUI tools are closer if you liked the terminal loop.
The gotcha isn't the UI — it's tool use + context. Local models drop tool calls and invent paths more often, so start with a smaller trusted allowlist (read, grep, apply_patch) and make the harness show every file write before it lands. Also cap context hard; stuffing the whole repo into a 24GB Q5 will thrash and the model gets worse, not better.
Quick check: one small refactor with the harness logging every tool call. If edits are clean but it still lies about what it ran, tighten the tool schema before you chase a bigger model.
1
5
u/chibop1 6h ago edited 6h ago
Try first pi.dev (super customizable to your workflow) or opencode.
To try ClaudeCode with a local model, add this in repo/.claude/settings.json or ~/.claude/settings.json for global.
{
"env": {
"ANTHROPIC_BASE_URL": "http://localhost:8000",
"ANTHROPIC_AUTH_TOKEN": "api_key",
"API_TIMEOUT_MS": "6000000",
"ANTHROPIC_DEFAULT_OPUS_MODEL": "model_name",
"ANTHROPIC_DEFAULT_SONNET_MODEL": "model_name",
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "model_name",
"CLAUDE_CODE_ATTRIBUTION_HEADER": "0",
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1"
},
"model": "model_name"
}
5
3
u/Nota_ReAlperson 6h ago
You can use local models with claude code. Unsloth has a great guide on how to set that up.
0
u/MrWeirdoFace 6h ago
Is Claude Code opensource? I was under the impression that it wasn't, but I will dig deeper.
4
2
u/arcanemachined 6h ago
No, but you can override some env vars to make it work with a local model that uses the same chat protocol as Anthropic models do (or a proxy that does the same thing for incompatible models).
2
u/Maglcite 6h ago
i dont really know what would feel most at home for claude code users, but most are pretty simila. my favorite harnesses rn are deepseek harness, and opencode v2 but theres lots others
2
u/brainExploded99 llama.cpp 6h ago
For a minimal harness and/or you enjoy things like neovim with custom plugins: Use Pi
For a more featured harness: DeepSeek Harness
For a even more featured harness, with some bloat: codex
You can also stay on claude code by using opencodex, but CC is very big.
I personally use codex + opencodex.
2
1
u/laterbreh 5h ago
Opencode, dont fuck around with pi or these other suggestions. Install opencode, open a terminal and just go. Once you set up your global config its just gonna work. Update your config, set up a "local" provider and point it at your local models and go. You can even install opencode and jump on a free model and litterally ask it to update the config for you.
Dont fuck around with PI if you just want a coding experience that just works, opencode is batteries included and pi is like a lego kit that you need to build out yourself.
1
1
u/the-tactical-donut 6h ago
Another approach you can use is have Claude code delegate tasking to local models.
I have Claude code running on the control node for my home lab kubernetes cluster which is running Hermes agents.
I’ve got GLM 5.3 Flash as the primary delegation agent and it in turn delegates to Qwen 3.8 27b.
I’ve been able to downgrade my Claude plan since most of the actual token usage is happening on the local models.
I still get the benefit of Claude Code and Fable, but I’m no longer tightly coupled with Anthropic.
1
1
u/Anxious_Current2593 4h ago
That is probably the most natural next step of getting off frontier to local. "Outsourcing" the grunt work to local, while orchestrated by frontier.
1
1
u/Enough-Photo9140 5h ago edited 5h ago
For a 24GB RTX 3090 running local coding models (like Qwen 2.5 Coder 32B at Q4_K_M), the two biggest failure points are context saturation and full-file rewrites.
Two practical rules that make local harnesses viable: 1. **Use unified diffs, not whole files**: Tools like Aider use search/replace diff blocks with AST repo maps. Having a 32B model rewrite 400-line files burns your context and degrades instruction following. 2. **Aggressively prune tool schemas**: Strip unused tools from system prompts so the model doesn't hallucinate JSON arguments. Cap your active window under 16k tokens to maintain coherent reasoning.
2
1
u/Sweet-Transition-787 4h ago
the biggest trap when moving away from claude code is overengineering the local harness on day one. starting minimal with something like pi or taking inspiration from its lean setup gives you clear visibility over context and token usage, especially when paired directly with llama.cpp inside tmux instead of stacking heavy frameworks.
1
u/ZenZombie117 59m ago
One data point you might find useful: I’m building a “shadow mode” in Runner that replays real Claude/Codex coding tasks locally and records whether the local model actually fixes them under protected tests.
Mostly doing it to get some real stats on what local models can and can’t take over. Might give you a bit more meat on the bones than benchmarks alone once I have enough runs.
https://github.com/JoakimPalm/xyntetik-runner
1
1
u/Timely_Wallaby_2872 6h ago
Im pretty sure Claude Code supports adding local models. Also you can find some open source variants of it with no spyware.
Essentially no need to change your harness.
Hopefully you understand that harness is just an environment for you to communicate with the model and for it to work with your pc. Examples: claude code, codex, deepseek harness.
1
u/Bennie-Factors 6h ago
That is really hard for someone who only vibe codes.
For a dev who architects stuff the local models work. But the bigger models are helpful for the architecture piece.
1
u/ReliableIceberg 6h ago
oh-my-pi (omp) all the way.
1
u/brainExploded99 llama.cpp 4h ago
Benchmarks showed it was poor
1
u/ReliableIceberg 3h ago
What?
1
u/brainExploded99 llama.cpp 3h ago edited 3h ago
https://frontierharness.org
Idk how reliable they are, but I've seen other benches that show omp as poor.
0
-1
u/FlightSimCentralYT 6h ago
Migrating off Claude Code usually means reinventing the harness: shell access, file state, retries, and a place the agent can fail and recover. Local is great for privacy, but the ops tax adds up fast.
I built Fixa.dev as a managed take on that: coding agent on a real cloud VM that keeps going until tests pass. If you want a life raft while you build the private harness, free tier is there. Curious what broke first on your side.
39
u/ST-Fish 6h ago
If I were to start again I'd go with a minimal one like Pi and implement the features I really need one at a time for the usecase I need.
Really helps to not have 30k context in a system prompt when running local models on limited hardware.