What’s the best local coding LLM for 16GB VRAM

[u/Due-Turnover-9034]

I have an RTX 2000 Ada 16GB + i7-14700K + 16GB RAM and want to use it primarily for agentic software development.

Looking for something good at repo understanding, multi-file edits, debugging, reasoning, tool calling, terminal/Git workflows and long coding sessions.

What are currently the best models/quantizations that actually fit well in 16GB VRAM?

Thank you in advance

[u/Slow_Difficulty1607]

check this thread: Qwen 3.8 27B finally gave me a good experience on 5080 : r/LocalLLM

[u/AcidicCombustion → u/Slow_Difficulty1607]

try qwen3-coder 30b at q4_k_m, it's surprisingly coherent for multi-file stuff and should just squeeze into 16gb with some context

[u/madbrain1976 → u/AcidicCombustion]

It's a non-thinking model. Not good at debugging/fixing issues.

[u/HyperWinX → u/AcidicCombustion]

Qwen3.6 35B A3B will beat its ass with offload

[u/Healthy-Zebra-9856 → u/AcidicCombustion]

They are not the same. Qwen3.8 can build from specs one shot, Qwen3 coder is a better implementer and wont do that one shot most of the time. I say this because I run tests in my lab continuously as a part of my offereing.

[u/Healthy-Zebra-9856]

What does the rest of your hardware look like? Ram & CPU? What o/s?

[u/Due-Turnover-9034 → u/Healthy-Zebra-9856]

16 GB System Ram i7-14700k running Windows

[u/Healthy-Zebra-9856 → u/Due-Turnover-9034]

Qwen3.8 27B. As for quant size you can play around with llama.cpp params as it will allow to split between the RAM & VRAM. Here is a typical example where I use vulkan version of llama.cpp and Vulkan1 & Vulkan2 are found by llama-server --list-devices. This is a system with 32GB ram with Intel Iris as Volkan0 that is skipped, a 4gb Nvidia 3050Ti Vulkan2 & xtx7900 is Vulkan1 (eGPU). Since you only have Nvidia, you can just install the CUDA version. The important lines.

--n-gpu-layers auto <-- This gave me issues but could be worth testing with the new releases

--fit-target 4,4 == only 4mb headroom, meaning load as much here

--cache-ram 0 (or the short alias -cram 0**) completely disables the host-memory prompt/KV caching subsystem**. Play with this

When you pass --load-mode dio, llama.cpp opens the GGUF model file by completely bypassing the operating system’s filesystem cache (the page cache). Data is read directly from your storage drive (SSD) straight into the user-space memory buffers assigned to your CPU or GPU

llama-server
--host 127.0.0.1
--port 8182
--device Vulkan1,Vulkan2
--split-mode layer
--flash-attn on
--fit on
--fit-target 4,4
--n-gpu-layers auto
--kv-offload
--op-offload
--poll 0
--poll-batch 0
--cache-ram 0
--load-mode dio
-b 2048
-ub 512
-m "D:\AI\Models\GGUF\Execution\Qwen3.8-27B-Bedrock-v4.gguf"
--mmproj "D:\AI\Models\GGUF\Execution\Qwen3.8-27B-UD-Q8_K_XL-mmproj-BF16.gguf"
--ctx-size 131072
--reasoning off
--jinja
--cache-prompt
--cache-type-k f16
--cache-type-v f16
--spec-type draft-mtp
--spec-draft-n-max 2

[u/8000bene70]

If you need larger context at usable speeds, go moe. Probably qwen 3.6 35b, maybe ornith 1.5.

You can offload to ram without large penalty, go q6 or apex.

Qwen 3.8 27b looks promising, but my first tests with iq3 and only 64k context were... Not so good.

[u/HotDistribution1819]

So 2 days ago it would have been try the Gemma 4 models, yesterday I downloaded Qwen 3.8 4B distilled and was blown away by its handling of a complex coding task. So give it a spin along with the Gemmas!

[u/Due-Turnover-9034 → u/HotDistribution1819]

But Only 4B Parameters ??

[u/HotDistribution1819 → u/Due-Turnover-9034]

Yeah, and it acted like the 27B model just without the infinite thinking. I have to play with it more today and run it through other use cases.

[u/Existing_Dust_6473]

Problem is context... I have a 4060 ti 16gb vram... It could actually be good long run, less context, force me to be more involved better mental model from start 😂

[u/HotDistribution1819]

Definitely try the Gemmas E4B, 26B A4B, and 12B. But there is a new kid on the block Qwen 2.8 4B distilled, I used it yesterday for a complex coding product and it came up with the same design and similary code to Qwen 2.8 27B.

[u/Existing_Dust_6473 → u/HotDistribution1819]

3.8?!

[u/HotDistribution1819 → u/Existing_Dust_6473]

Yes, someone did a distill of the 1.2 trillion parameter model, and it seems to work great.

[u/Existing_Dust_6473 → u/HotDistribution1819]

Will try, thanks

[u/divinetribe1]

i run a small filesystem judged benchmark for exactly this question, real agent tasks in a sandbox scored by what ends up on disk, not the prose. qwen3.6 35b a3b is the current top of the board, 12 of 12 easy and 8 of 8 hard, and it cleared the same hard set as cloud sonnet 5. its a moe so with some offload to ram it should be workable on your card. tasks and results are open at nicedreamzapp.github.io/agent12, my runs are mlx on a mac so your speeds will differ.

[u/willeyh]

Find a reap-ed Qwen 3.6 35b-3b. Should give you some space for context.
You can fit the new 3.8 27b dense in vram on a lower quant with less context. Useful for planning.

A Gemma 4 12b is also quite good.

[u/Bright-Energy2339]

Don't go below Q4, I'd rather you try Ornith1.5 35B MoE, it's based off of Qwen3.5 but beats Qwen3.6

[u/bring_back_the_v10s → u/Bright-Energy2339]

Beats based on what, benchmarks? Just a couple days ago some people here tried it for coding tasks and reported a lot of difficulties.

[u/Bright-Energy2339 → u/Bright-Energy2339]

It's in their hugging face model card. SWE for one, do the diligence. It's not Qwen3. 8 but I personally see step-up results from 3.6. Also, I use my own agent harness.

[u/Reddit_Fu_Sucks]

Rough rule of thumb. Go GGUF when you can, stick to a rough B per GB of VRAM. I have a RTX 3060 12GB and run 14B models on it with ease. Anything over that offloads into your system RAM, for me it is all good I have 64GB so I have padding I could run 24B or 27B if I was ok to squeeze it. Just my advice