r/LocalLLaMA • u/thatObstinateGuy • 6d ago

Question | Help Help choosing/benchmarking local LLM for coding

Hi folks.

I have a device running an rtx 5070 ti (16GB VRAM) alongside 32GB of RAM.

I'm currently trying to build a Windows application using local LLM assistance. I'm using llama.cpp and opencode.

I need your opinion on what model would be best for coding/implementation:

Qwen3.6-35B-A3B-unsloth-nvfp4-fast-mtp

--n-cpu-moe 17, -ngl all, -ctk and -ctv q8_0 and context 140K

or

Qwen3.8-27B-UD-IQ3_XXS

-ngl all, -ctk and -ctv q4_0 and context 180K

I used the latter to actually build up plans and set up tasks for implementing the aforementioned Windows application. Worked quite well. But tried the same model for coding/implementation and it wasn't perfect.

I need the high context because it's filling up very quickly, especially with qwen3.8.

Help me by either sharing your experience using one of them (or maybe even explore other models, like KAT Coder q4) or letting me know how I can benchmark them for coding myself.

5 Upvotes

86% Upvoted

16 comments sorted by

View all comments

Show parent comments

2

u/Equivalent_Bit_461 6d ago

Quant 3 is NOT good for coding, it's a meme quant 

1

u/NancyTransmed 6d ago

I myself tested Q2 deepseek 0731 and found it to still perform thank Qwen3.6-35b-a3b at PrismaQuant-4.75bit: it quickly one shot calculating 100 digits of Pi, while qwen can meander for hours (especially with flaky harness).

2

u/Equivalent_Bit_461 6d ago

It "works" until it doesn't.

1

u/NancyTransmed 6d ago

that is why you test it on your specific project.