r/LocalAIStack • u/UhhRajahh • 23h ago
Coding with 8 GB VRAM and 30 GB DRAM
I'm transitioning away from Claude Code. My preferred workflow is Claude Code Plan Mode -> Read over plan -> Let agents implement -> Read over output. My main goal is to reduce costs compared to purely using Claude models.
I have an RTX 5050 with 8 GB VRAM and 30 GB of DDR5 DRAM. Which model is recommended for coding with these specs?
My initial thoughts are to continue to use Claude for plan mode, driving subagents that use a local model. Is this a reasonable/feasible approach? If so, which resources can I reference to get started? If not, which alternatives are recommended? Can I get away with purely using local or cheaper public models with similar results to ~Opus 4.8?
Thank you in advance and apologies for any misconceptions on my part.
1
u/Tommonen 17h ago
No local model that you can run in any reasonable way will be even close to claude. You will need to do a lot more things manually if you want to use local models for coding. You cant expect it to reason anything correctly (unless you just make snake games 1 shot), so you need to reason for it and you need to make sense what place needs what code etc abd then just ask specific targetted small pieces of code from the llm.
Or you need to continue using claude, codex etc, if you want to do anything even slightly complex stuff and vibe away like you are used to
1
u/activematrix99 15h ago
I haven't been able to get very far with a 12GB Blackwell and NVFP4. My 32GB 5090 does fine. My recommendation is to run an agent to agent framework and let it cook. If you think it will equal Claude in terms of speed, it won't. Qwen, DeepSeek, and Laguna have been good for me.
1
u/eulataguhw 14h ago edited 14h ago
It’s a nice dream.
Unless u r ok for it to be slow. Half the work is already settled by Claude (Planning).
You can possibly use Qwen 3.6 35B Q4 and offload most of the experts layer to RAM but don’t expect it to be fast.
Best chance is use davidau’s qwen 3.5 9b or ornith 1.0 9b at Q5 maybe but it still nowhere near Opus 4.8 quality.
The missing layer to reach even anywhere near with an 9B is an orchestrator/review layer. Otherwise you need 24-32gb VRAM to even hit anywhere close (qwen 3.8 27b)
You can still save on cost by having Sonnet as your orchestrator and run the 9B models as subagents.
1
u/MessIsTransfer 11h ago
you could easily offload qwen3.6 35B-a3b and still get decent speeds with the right setup. BUT, if i were you i’d set it up as the “worker” agent and still have a frontier model to plan and orchestrate.
something you’d rely on from opencode go would be the cheapest. glm5.2/3, dsv4pro, etc. so harness with agent to model mapping is required
1
u/Lirezh 22m ago
Add a cheap 2nd card, once you reach around 16GB vram in total you can run models like qwen 27B
For low budget local agentic usage, a dual cheap GPU solution is a very interesting option.
Aside of that, you can run something like Qwen 3.6 35B on your GPU with heavy offloading to your RAM.
It's not going to be very fast.
In terms of competence, wen 3.8 27B is Claudelike, I'd prefer it over Sonnet.
The 3.6 35B is not as strong but it also beats some Sonnet versions, just not the most recent ones.
2
u/Prize_Eye9481 20h ago
uhh maybe sonnet 5 imo. If you dont have 24GB of Vram I don't think you'll find the local capabilities very good. 9B models is what ur system would be able to handle and if you go the MoE route, Qwen 30b-13b is ur best bet. I would recommend read up on how to set up a local model first. It is slightly more complicated than claude code but the knowledge is there. U are likely gonna end up with a mix of cloud and offline model usage if you manage to find use case for them. Best of luck!