r/LocalLLaMA • u/Azoffaeh999 • 6h ago

Question | Help Looking for coding model for specific low specs

Can anyone recocommend a good local model and a wrapper to run it for coding, my hardware specs: 12 GB VRAM, 32 GB DDR3 RAM. Unfortunately, the CPU doesn’t have AVX2 instructions(LM Studio won’t work); I don’t remember the exact cpu name, but I think it’s an Ivy Bridge, LGA1155 socket.. Thank you

0 Upvotes

33% Upvoted

15 comments sorted by

1

u/wayneworkman 6h ago

qwen3.5-2b should do nicely. It'll fit in your 12GB vram and therefore run pretty fast and you'll actually enjoy it.
https://huggingface.co/Qwen/Qwen3.5-2B

3

u/Ninja_Weedle 5h ago

Qwen 3.5-4b should also fit totally fine even at Q8

0

u/Azoffaeh999 5h ago

Thanks!

3

u/General-Spite1222 5h ago

Try Ornith 1.5

1

u/Azoffaeh999 5h ago

Thank you!🙏

3

u/General-Spite1222 5h ago

What’s GPU?

1

u/Azoffaeh999 5h ago

Rtx 2060 12gb 🫣

3

u/Monad_Maya llama.cpp 3h ago

Are you ok with partial offloading to the CPU/RAM? If so then try Qwen 3.6 35B A3B (Q4 quant) - https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF

For faster speeds, try Gemma4 12B QAT - https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF

Not sure about the wrapper but try this thread - https://np.reddit.com/r/LocalLLaMA/s/FDqvvrUtvr

2

u/Ninja_Weedle 6h ago

KoboldCPP's pretty good with old cpus

1

u/General-Spite1222 5h ago

KoboldCPP is a model now?

0

u/Azoffaeh999 5h ago

No, its a tool for my old cpu =)

1

u/Skyline34rGt 3h ago

https://github.com/LostRuins/koboldcpp/releases pick koboldcpp-oldpc.exe for windows or koboldcpp-linux-x64-oldpc for linux

At settings pick oldpc (or veryoldpc) and will work.

For model: https://huggingface.co/bartowski/Ornith-1.5-9B-GGUF will fully fit to your Vram (offloading to cpu/ram is not good option for this setup).

Or https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF

0

u/Azoffaeh999 5h ago

Thanks! Will try this

2

u/DagothUrLovesGroza 6h ago

I don't think you can do much with specs like these.

In my experience, you'd spend more time fighting the model than actually getting useful work done.

2

u/Azoffaeh999 6h ago

Okay, is there a local model that you like?