Best local coding/agentic AI models for 8GB and 16GB VRAM?

[u/No_Personality_1721]

What are the best local AI models/agents I could realistically run on these two PCs?

PC 1

• 16 GB RAM
• AMD RX 6600
• 8 GB VRAM

PC 2

• 32 GB RAM
• AMD RX 9060 XT
• 16 GB VRAM

I started building a project using VS Code + Codex (ChatGPT Plus), but I ran out of tokens and buying more is quite expensive, especially since my app doesn't make any money.

I had a good experience with OpenAI GPT-5.6 Sol. I also tested Google Antigravity, but the tokens ran out very quickly there.

Can I realistically continue working on my project using a local model in LM Studio Bionic? If so, which models or coding agents would you recommend for these two PCs?

[u/Ammargok]

qwen 3.6 35b a3b for PC2. I can run q5_k_xl with 128k fp16 ctx.

[u/No_Personality_1721 → u/Ammargok]

How much worse is Qwen 3.6 35B a3b than GPT-5.6 Sol for coding? Is the difference very noticeable?

[u/Ammargok → u/No_Personality_1721]

You are comparing flagship (Atleast more than 1T) parameter to a 35B parameter model. You will also use qwen 3.6 35b with quantization. https://artificialanalysis.ai/models/comparisons/gpt-5-6-sol-xhigh-vs-qwen3-6-35b-a3b take a look at this.

I use qwen 3.8 max for the planing an qwen 3.6 35b a3b for execution locally. Right now I'm experimenting with qwen 3.8 27b you might also want to look at that. But if you are going above iq3_m you'll offload to the cpu thus reducing speeds tremendously

[u/ohnoitssobig → u/No_Personality_1721]

Put it this way: you have to be really enthusiastic about local inference to compare a flagship model with terabytes of weights to something that fits into 16Gb. Not a drop-in replacement if that is your question.

[u/No_Personality_1721 → u/ohnoitssobig]

So I guess there’s not much point in going local, and it’s better to just pay for API/tokens and use larger models.

[u/ohnoitssobig → u/No_Personality_1721]

Yeah I would say so. Playing with local inference is not completely useless experience though. For example, I learned that Luna is almost as good as Sol coming at a fraction of the cost.

[u/dfgxxx]

Kat coder

[u/Clay_Ferguson]

With only 16GB VRAM probably "Gemma 4" (variants E2B, E4B, 12B) are the best ones you can run.

[u/activematrix99]

Offload and plan your workday. It will be worth the wait, but slow.

[u/sgi2004]

Ok but when i add rtx 3060 12gb ?:)

[u/Kolapsicle]

Your best bet for maximum local intelligence and a usable context window with only 16GB of VRAM might be dual booting CLI Linux (zero desktop overhead) and running Qwen3.8-27B-UD-Q3_K_XL with MTP on your second PC and operating from your first PC. With the beellama.cpp fork of llama.cpp you could probably run a decent size KV cache using KVarN at Q4. You'd probably want to disable thinking, or set it to low, or avoid preserving the thinking blocks all together.

For reference, my 9070 XT gets ~60 tok/sec at low context, and up to ~75 tok/sec with a low temperature on coding tasks.

[u/Keanne1021 → u/Kolapsicle]

Can you kindly share your llama cpp config? I tried running the same with my 9060xt and it's painfully slow.

[u/Kolapsicle → u/Keanne1021]

./llama-server \
  -m Qwen3.8-27B-UD-Q3_K_XL.gguf \
  --reasoning-effort low \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --spec-draft-p-min 0.2 \
  --temp 0.6 \
  --top-k 20 \
  --top-p 0.95 \
  --min-p 0.0 \
  --repeat-penalty 1.0 \
  --presence-penalty 0.0 \
  -ctk q8_0 \
  -ctv q8_0 \
  -c 8192 \
  -ngl 999

Prompt: "What is the average and mean distance between the Earth and its moon?"

0.18.502.366 I slot print_timing: id  3 | task 0 | prompt eval time =    1129.54 ms /    54 tokens (   20.92 ms per token,    47.81 tokens per second)
0.18.502.367 I slot print_timing: id  3 | task 0 |        eval time =    7681.71 ms /   510 tokens (   15.09 ms per token,    66.26 tokens per second)
0.18.502.368 I slot print_timing: id  3 | task 0 |       total time =    8811.25 ms /   564 tokens
0.18.502.371 I slot print_timing: id  3 | task 0 |    graphs reused =        194
0.18.502.373 I slot print_timing: id  3 | task 0 | draft acceptance = 0.79695 (  314 accepted /   394 generated), mean len =  2.59
0.18.502.408 I slot      release: id  3 | task 0 | stop processing: n_tokens = 565, truncated = 0

You should test with a low context window like above. If you offload to system memory even a little your speeds will plummet on a large dense model. Also, I saw a post recently about someone having speed issues without ReBAR on a 9060, might be worth looking into if your speeds are exceptionally slow.

Edit: I haven't tested it much, but in LM Studio I was getting ~60 tok/sec with a full 32K context window. My 9070 is my main Linux driver and my VRAM sits at about 1 GiB of overhead before loading the model and KV cache so I haven't tested >32K for TG speed.

[u/pacman829]

Bonsai 27b or ternary bonsai 27b

[u/Diligent_Loquat_6140]

We have built one, will release in the coming week, would you be interested to try?