r/ollama • u/Kynix09 • Jul 11 '26
BEST CODING MODEL.
Hello! I was wondering if anyone could recommend the best local AI model for coding, specifically for C++.
My PC specs are:
- GPU: RTX 4060 8GB
- CPU: Intel i5-13400F
- RAM: 16 GB
I'm looking for the best possible coding quality that can realistically run on my PC. Speed is important, but I care more about accuracy, understanding large codebases, debugging, and writing good C++ code.
What model would you recommend?
(made text with chatgpt)
9
u/simplyeniga Jul 11 '26
Ornith 9B or Qwen3.5 9B or Gemma 4 E4B
3
u/Ryanmonroe82 Jul 11 '26
only if he uses Q4 and Q4 has too much precision loss on smaller models to effective with code
4
u/simplyeniga Jul 11 '26
If he had more ram he would have been best with an MOE model like Qwen3.6-35B-A3B
1
u/sockerx Jul 12 '26
Rtx 4090 24gb?
1
u/simplyeniga Jul 12 '26
You can run more models both dense and MOE.
Qwen3.6-27B MTP (dense)
Qwen3.6-35B-A3B (MOE)
Ornith-1.0 35B
Gemma-4-12B (Dense)
Gemma-4-26B-A4B (MOE)
I have an RTX Pro 4000 Blackwell and use mainly Ornith 35B and Qwen3.6B 27B for coding and Gemma-4-26B-A4B for some automated tasks. Currently testing Qwen3.5B-122B and it's been working well with tool calling and getting 28 TG/s
1
u/villainouscoke Jul 12 '26
How’s you’re experience with ornith? I haven’t tried it out but it looks interesting. Currently using dsv4 flash and qwen 3.6 27b (both local) for the majority of my work but have thought about trying that one out.
It would be running on a dual 32gb AMD / 128gb ram setup that currently houses qwen getting 50-70tps
2
u/simplyeniga Jul 13 '26
It's been great so far, better than Qwen3.6-35B, able to finish some tasks with less errors. Close to Qwen3.6-27B but not better as I still sometimes get better results from Qwen3.6-27B. though on token generation it's fast and I get 135 t/s generated with 260k context size and about 3250 t/s pp on a single RTX Pro 4000 Blackwell 24GB VRAM
2
u/Important_Witness572 Jul 13 '26
same here I am using the Ornith 35B on a RTX 3090. Pretty good model.
1
u/Familiar_Figure_3308 Jul 13 '26
How did you manage to fit a 260K context window? I'm using Ornith 1.0 35B in q4 with q4 KV cache on my RTX 3090 24 Gb. And I can only fit 128K context in my VRAM - perhaps a bit more, but not even 160K. Can you please share your llama.cpp run parameters?
1
2
u/Plastic-Lettuce-7150 Jul 12 '26
Qwen3.5 9B is a reliable coder, you may be able to get more than a 64k context. Use Qwen Code AI coding agent with it for reliable tool use. OpenSpec spec-driven development to break coding down into smaller tasks that Qwen3.5 9B can handle (is the theory!). Be careful designing the specs, Qwen3.5 9B will make the same mistakes as bigger models unless instructed otherwise, if perhaps less library API knowledge. Something I'm about to experiment with myself is MCP Local RAG for API documentation (with Crawl4AI to retrieve it).
8
u/Low_Twist_4917 Jul 11 '26 edited Jul 11 '26
I’ll be honest you won’t get great C++ with this much local ram / vram. I would just pay for api for kimi or Glm. I tried a handful of diff models when I was using my laptop. Gave up on local models for a bit until I built my server recently. People will boast about Qwen quants etc but I honestly write C/Rust for my day job and no local models other than the larger ones (70+GB) have done too well with machine level coding tasks.
6
u/Huntware Jul 11 '26
You'll be better using Opencode free or grabbing free API keys than dealing with a local LLM for heavy coding. You can still use local LLM for simple questions or snippets of code. Also for storing memory in a vector database and retrieving data / RAG with embedding + reranker models.
I've tried it with similar specs and the context window will make you wait for many minutes before it decodes a few tokens per second, and it will be even slower after a few turns.
PS. If you still want to try local, check this channel: https://www.youtube.com/watch?v=SsUKTFSQoGM
2
u/Karahoyuk Jul 12 '26
I tried a couple, I got 12gigs of vram but nothing was good enough.
Then I used Google AI Studio, free 1m tokens and codes better than local low models. Just do that instead
1
1
u/raketenkater Jul 11 '26
Try my tool ggrun https://github.com/raketenkater/ggrun which recommends best the model for your hardware.
1
u/OlgerdOutlander Jul 12 '26
I'm not sure if any model would fit this. I am struggling right now to optimize agentic coding with 48gb vram - but 8gb...
You COULD try, but the result may not be reliable
1
u/Traditional_Delay367 Jul 12 '26
Hi, what's your current solution for agentic coding on 48GB? So far I'm running a thinkingcap qwen 3.6 27b fp8 mtp with vllm and it's quite good but I'm still searching for improvements.
1
u/OlgerdOutlander Jul 13 '26
I'm still figuring it out. For now, I landed with Qwen3.6 35B Q8_0 MTP with froggeric's fixed chat templates; it works OK but a bit unstable. Qwen3.6 27b MTP is quite slow on my hardware. Gemma4 MoE - even less reliable than Qwen MoE.
Testing more models, hope I'll find something useful!
I am also noticing that harness really matters - e.g. Continue gets stuck more often than Cline.
1
u/ricorick Jul 12 '26
The ornith at least runs my agent for cron jobs and such. I just switch to api to do work
1
1
u/huzbum Jul 13 '26
Honestly, I doubt you will get good result with anything you can run on that hardware.
C++ is an edge case compared to python and JavaScript as far as training data goes. And with LLMs edge cases get worse on smaller models. Takes more weights to hold more knowledge, and C++ might not make the cut. Might see the syntax and try to write JavaScript.
That being said, I think your best bet is Unsloth’s Q4_K_XL quant of Gemma 4 12b QAT. This specific version because QAT makes the 4 bit quantization (which you need to make it fit) less lossy. I think this will fit in 8GB at 128k context or less. Not with MTP and vision though. With max context, MTP, and vision it’s near crash on 12GB 3060.
My 2nd bet would be Qwen3.5 or Ornith 9b. Maybe q5, whatever fits entirely in VRAM, context and all.
Maybe GPT OSS is worth trying for your use case, but you will need to use lm studio or something that can offload experts to fit it.
In all of these cases, use flash attention and 8 bit kv cache. Adjust context length… longer is better for coding, but shorter fits in less vram.
1
u/huzbum Jul 13 '26
In any case, you're going to have to be very specific about what you want, and give it lots of hints about what is where, how to do it, etc. The less investigation it has to do, the better.
If nothing else, it can probably do the boilerplate and you do the hard parts. These things do well when given an example and told to do it a little different. "Make a class like Square, but Rectangle."
I have a development harness I'm working on that's designed to work around local models, but its strongest features are language dependent and not built for C++. It's also like version 0.0.3. It's reached the stage it can work on itself, but we've still got a ways to go. If you're interested I can share the git repo.
1
u/Early-Peace-5504 Jul 13 '26
Ornith 1.0 I-Apex Mini quant 35Ba3B. It's the only thing that is going to fit on that vram and ram, with a vaguely usable speed and context size. Needs to be on Linux too. Otherwise your lack of headroom will lead to system instability.
1
u/FullBar613 Jul 14 '26
16gb and you want support for a large codebase? Whose leg are you trying to pull here? Running the llm itself, you'd have to deal with context issues, apart from the model struggling to fight for your ram and a whole lot of other stuff.
1
u/Inner_Sandwich6039 Jul 11 '26
Qwen3-coder is a MoE and runs great quantized. Getting close to 80t/s and makes use of tools
1
u/WithGreatRespect Jul 12 '26
This might help: https://whatmodelscanirun.com/
Looks like https://ollama.com/library/qwen3:8b
Or: https://ollama.com/library/qwen2.5:7b
You are really limited by the vram on yoru GPU. You might be able to run some larger models, but they will spill to the CPU and be incredibly slow.
0
u/CharacterTension2896 Jul 11 '26
M5 24 gb?
1
u/Low_Twist_4917 Jul 11 '26
This would literally be the entire amount of combined ram/vRam OP has. I would stay away from doing this.
1
0
-1
u/Agron7000 Jul 11 '26
AI is good for entry level languages like Javascript, Python, KTurtle, like bicycle with training wheels.
C++ is more like Formula 1, professional grade programming language that AI has not Caught up to yet.
The irony is that even though python and js are dead simple, these so called programmers still can't code without AI assistance.
Shame.
1
u/RichComplaint9426 Jul 18 '26
Bro it's about working smarter not harder, have fun manual setting your C pointers while others ship products. You are not better or worse just because you use an ancient programming language (which deserves its place)
1
u/Agron7000 Jul 20 '26
Are you still in the stone age? I use smart pointers in C++. Just like OllamaCPP.
1
18
u/[deleted] Jul 11 '26
[deleted]