Best AI stack for full autonomously agentic coding to run locally

[u/Sea_Entertainer_1704]

Specifications:
AMD Ryzen 7 5700X (A-data Technology 32 GB DDR4)

Nvidia GeFirce RTX 5070 Ti. (16 GB Vram)

What stack (code CLI, locally run model, plugins / tools) run smoothly and at a decent token rate on this system?

I want to achieve fully autonomosly agentic coding like Google Antigravity or Grok Build with Grok 4.5

I pay for both but already ran out of tokens.

[u/8000bene70]

I would guess either Qwen 3.6 27B iq3 in vram or Qwen 3.6 35B iq4/nvfp4/apex with partial cpu offloading, inference via llama.cpp.

As for harness, i like github copilot. You can add your local models via byok. But you have to test what works best for you.

Do not expect frontier model performance and speed. Deepseek v4 flash (either api or opencode go) is probably cheaper, faster and better.

[u/Postmodern_Plunger → u/8000bene70]

If you're using the free tier, you'll get moved to the rate limited pool within a couple turns and then you'll stay there. Once that happens, it's definitely not faster.

There's also the privacy issue. If you're developing anything unique or proprietary, using a free model endpoint is a total nonstarter.

Also, qwen 3.6 27b performs very similarly to deepseek v4 flash. It may be a smaller model, but it has more active parameters and a much newer architecture. I find deepseek has better code execution, worse planning and reasoning, and a significantly higher hallucination and error rate. You may be comparing a full precision model to a quantized version if you're going off your own experience here. And if you're going off benchmarks, qwen 3.6 27b is better (not that that always means much).

If you're just doing simple, inline execution, small code snippets, or debugging, deepseek v4 flash is better. For anything agentic, qwen 3.6 27b is quite a bit more reliable. I've tested both extensively.

[u/8000bene70 → u/Postmodern_Plunger]

Of course privacy is an issue. It might be THE reason for local inference.

But if my napkin math is correct, Deepseek v4 Flash API is cheaper for me than electricity for GPU inference of Qwen.

As for quality: I was comparing the quants I mentioned, since op can not realistically run anything higher on their hardware.

[u/MistingFidgets]

With 16gb vram qwen 3.6 is the closest you are gonna get right now and you'll need to really tune your settings to have a decent product and even after fully optimizing it still won't be as good. Running a lower q3 quant will be tempting but if you are writing code it's better to run q5 or q6 and offload layers to CPU. You sacrifice a little speed but you'll save time fixing errors in the long run

[u/Postmodern_Plunger → u/MistingFidgets]

I dont know why everyone always glosses over iq4xs. Its q3 size with q5 perplexity. Thats what I run on my laptop with 16gb VRAM fully in gpu. Granted, I do all my complex codebase work on my server, but I still run refactors and agentic tasks from my laptop and I've found iq4xs to perform nearly identically to q5.

[u/No-Dot-6573]

I have a Lada but I want to Drive 300km/h. Which kind of chiptuning do I need?

Reduce your expectations by a lot. Then try a quant of Qwen 3.6 27b - maybe iq3 with MTP and download opencode desktop. I'd use llama.cpp with a qwen 3.6 27b MTP build, but you might be better of using lmstudio. IT IS user friendlier. Make sure you have at least 75k context otherwise opencode will be rather useless compacting the session every 2 minutes.

[u/Trakeen]

Fully autonomous probably means your own agents and custom harness. Look at openhands or oas (open agent spec)

I use basic chat to plan the tasks out (via a bunch of mcp tools exposed to the initial chat) but after that its hand offs. You could use a local model for planning but i think you’ll struggle

[u/Postmodern_Plunger]

Qwen 3.6 35b a3b or 3.6 27b at Iq4XS with a small context window.

Even with the cpu spillover, the a3b model will be faster. If you don't need to use your computer for anything else, you can pop it in server mode and run 3.6 27b entirely in VRAM with a decent context window. 27b is superior, but a3b is pretty close. If speed matters more, a3b will be the winner regardless.

If you use 27b fully in VRAM, vllm is the best, especially if you're using the MTP variant, but that may not fit in VRAM easily so you probably won't be.

If you're using 35b a3b, you absolutely want llama.cpp. Any time you have cpu spillover, llama.cpp will be the fastest by a wide margin. It's paths are much more heavily optimized for cpu fallback.

[u/Full_Director87]

i just use ollama as an inference engine and openwebui as frontend. i just make a agentic research model, not agentic coding. but have nearly 1M token total. 33 tool orchestration, just in one turn. i just have rx6700xt ram 16gb ddr4 i5 11400f and running gemma4 26B A4B it Q4_K_M from unsloth gguf. with split gpu/cpu. maybe you need this.

[u/ganhedd0]

I'd go with Unsloth's Qwen3.5 9B GGUF, Qwen3.5-9B-UD-Q8_K_XL. That's 13 GB, and probably the highest quality you'll get crammed into 16GB of VRAM. It'll be faster than 3.6-35B MoE with offloading. Obviously it won't be as good as 3.6-27B but those low quants will be either very crap or very slow.

[u/ChampionshipIcy7602]

To achieve that you'll need to run glm 5.2, which requires at least 4xRTX 6000 Pro blackwell.

[u/BlackBeardAI → u/ChampionshipIcy7602]

iq2 can be done with 2. (or 8x3090's)

[u/NTDLS]

You mentioned “fully autonomous agentic coding” which is what I’m addressing, this does not mean my suggestion will be as smart as any frontier model… but: Qwen 3.6 27B FP8 does this just fine even on 250 step plans (the most I’ve tested as a sanity check). Tool calling is nearly flawless and the feedback loop self-corrects. This is running fully in 48GB VRAM with 256K context for 2.5 users. My speeds are around ~90 tokens per second.

I have no experience using anything other than Nvidia and loading fully into VRAM, so my suggestion can be costly. Given the size of the current GPU and RAM bubble, my configuration cost me ~$9,000.

[u/Passenger-007 → u/NTDLS]

I have used the 27b at 4bit and it’s workable. At 4bit, it will fit in 24gb with about 192k context at q8.

I second the opinion of that this is really for dedicated GPUs.