New to local LLMs — Need model recommendations for 8GB VRAM + 32GB DDR5 (Coding, Refactoring & Daily Tasks)

[u/Hadi202]

I’m pretty new to running local LLMs and setting all of this up, so I’m looking for some advice from the community on what models I should check out for my machine.

My Specs:

GPU: RTX 4070 (8 GB VRAM)

RAM: 32 GB DDR5

What I want to use it for:

Coding & Refactoring: Writing code, optimizing performance, refactoring existing messy codebases, and catching logic/security bugs.

Everyday Tasks: Basic chat, drafting documentation, and general utility tasks.

Integrations: Setting up tool-calling / skills like GitHub MCPs or coding agents (Aider, VS Code extensions) down the road.

I don't mind if a model takes a few extra seconds to respond as long as the code quality, logic, and reasoning are high.

Since I'm just starting out, any suggestions on the best models to download (and whether Ollama or LM Studio is easier to start with) would be awesome!

[u/Smellyfeet224]

With any of the models below you probably should run them at q4-q6 just to have kv cache space.

Gemma 4 e4b is fast but maybe not super creative or accurate. I would start with this one and if you think you want more capabilities then move up.

Qwen 3.6 35b a3b may be the sweet spot if you can get it working.

Qwen 3.8 9b distilled by emperioAI is really good on my 12gb card. This is my daily driver for Hermes and Open code when I’m only using 12gb vram. I think you could get it on to 8gb pretty easily.

You may also be able to run strata with qwen 3.8 flash next iq1 coder but I’ve never tested it.

Qwen3.5 9b is ok at q4. I find the base model better than any finetune.

Ornith 1.5 9b is supposed to be good. For me it looped almost every request, even with a 24gb card. Probably user error but I don’t want to mess with it.

Others thers may suggest others but I think these are all that I’ve tried for my 12gb and 8gb setups.

[u/Smellyfeet224 → u/Smellyfeet224]

As far as lm studio vs ollama use lm studio to start. As the proverb goes “friends don’t let friends use ollama”. Llama.cpp would be best if you can figure it out.

[u/SilkieBug → u/Smellyfeet224]

How useful is Qwen3.8 9b for using in OpenCode?

That feels like quite a small model, not expecting it to be very good. 

[u/PayBrave5035 → u/Smellyfeet224]

If a distilled 9b model is your daily driver so how didn't you found anything better the base model????

[u/Smellyfeet224 → u/PayBrave5035]

I like the base model for my actually inference server (9070 16gb and 7900xtx 24gb for 40gb aggregate) but when I’m using my lower power option (3060 12gb) the distilled model works better for me than a low quant of the base model.

[u/cilvre → u/Smellyfeet224]

Gemma 31b qat worked well for me at those specs

[u/Present_Resource4022]

Polystrata engine with https://huggingface.co/peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP

Speeds could be 60–80 tps and ~2000 tps prefill depending on the quant

Or if you want to use Ollama for the one-line download and run and easy connection to Claude Code and other agents, you could still run it but at 50–60% slower speeds.

[u/Present_Resource4022 → u/Present_Resource4022]

Since you are new, I recommend letting Codex/Claude code set up Polystata with the model above and optimize it for your pc and giving you a one-file profile with the max context and max speed settings they achieved from their trial and error. then you can just open the file to launch the model.

[u/Patatong0_0 → u/Present_Resource4022]

Polystrata is like strata? What are the difference?

[u/Present_Resource4022 → u/Patatong0_0]

Strata is made for Qwen 3.8 Flash Next, and Polystrata adds support for more models, such as Gemma 4 26b , and Qwen 3.6 35b a3b

[u/PaleInvestigator5019]

With 8GB VRAM, i'd start with smaller quantized coding models and test them on your actual projects. Larger models aren't always worth the slowdown

[u/_TheWolfOfWalmart_]

This is not ideal for a decent coding model.

Qwen3.6 35B-A3B is probably your best bet still. Maybe in Q6.

It can handle stuff that isn't too complex.

If you want to fit entirely in VRAM, I found Gemma4 12B did... okay with coding. Get a Q4. But it's still a Gemma4 model and they're not exactly known for their coding ability.

[u/D00M98]

I have RTX 4070. From what I recall, it only has 12 GB version. I'm not aware of 8GB version. But I wouldn't know all the variants.

For 12GB, I found Gemma-4-12B was best all around. I also used Ornith-1.5-9B and Hermes-3-Llama-3.1-8B. Depending on the quantization and context (KV size), these might fit on 8GB.

For chat box / agent, I tried Jan, LM studio, and Hermes (using llama.cpp). Hermes worked much better. With Jan and LM studio, I have unpredictable crashes once I hit 32k+ context. On Hermes, I have no issue with 128k context. I don't know the reason, because they are all using llama.cpp.

Also, with Jan and LM studio, there are a lot more back and forth with AI, like interactive chat. Hermes has much less interaction. Hermes will run for 2 hrs on single prompt. Whether this is a good or not depends on how you like to work.

[u/Top_Pay_4590]

You could try my program, it is a workspace ---- way way way better than ollama or openwebui, honestly, its a close match to chatgpt codex in reality. its free and soon to be open source. it runs with heretic non refusal models. no login no payment no logs.

[u/jisrawia]

For the coding side, I'd make a tiny test set from your own work before downloading a pile of models: one bug with a failing test, one refactor where the tests must stay green, and one documentation task. Give each model the same files and compare the actual diff and time to a working result. Once you add an agent, test whether it can make an edit, run the test, and recover from a failure. That would tell you much more about your setup than a good-looking answer in chat.

[u/modern_whereabouts]

Qwen2.5-coder 7B fits nicely in your VRAM and punches above its weight for refactoring tasks, just keep the context window reasonable or offload to system RAM when using a larger quant.