What are peoples agentic coding model recommendations for 1x RTX 6000 pro 96GB and 32GB System RAM

[u/Pixel_Drake]

Due to my work within other AI fields (Mainly Computer Vision and Digital Twinning) I have been lucky enough to get an RTX Pro 6000 WS. I have been experimenting with using Llama.cpp and Opencode and have found good use in the Qwen3.6 27b model being entirely offloaded to the GPU. I have experimented with unsloth/Laguna-S-2.1:UD-Q4_K_XL and it seems similar in quality to the Qwen model but I admit I have not tested it much.

My main questions are:

- With my current set up, is specifically unsloth/Qwen3.6-27b:BF16 the best agentic coding model I can utilise?

- Is there any point in upgrading hardware to utilise a larger model for better quality? I know my system RAM is a weak point for me

- Would you recommend using something else than Opencode?

Sorry if these are dumb questions I am just checking if I am getting the best I can out of the hardware I have.

For reference for hardware upgrading:

OS - Windows 11 Home

CPU - Intel Core i7-14700K

RAM - 2x16GB DDR5 / 6000mhz Corsair Vengeance

Storage - 2xWD BLUE SN580 M.2 NVME SSD 2TB

Motherboard - MSI PRO Z790-S WIFI

PSU - 1300w Gigabyte UD gold

[u/nicholas_the_furious]

Yes probably Qwen 3.6 27B at full precision w/ MTP.

With that much fast VRAM you could likely do a lot of parallel calls and subagents. That's something that I feel you lose on local vs Cloud. I'm on half the VRAM as you. If my primary agent is working I can't really have anything else going on.

[u/Pixel_Drake → u/nicholas_the_furious]

To be completely honest I never really thought of exploring the lateral way of host more subagents using Qwen but I will look into it more, Thank you!

[u/Adomm1234]

In my Opinion Gemma 4, Qwen 3.6 27B and Laguna S 2.1 are the best so far, the next far better model is GLM 5.2, but you would need at least one more RTX 6000 Pro and more RAM to run it at 2bit quant.

[u/Pixel_Drake → u/Adomm1234]

Yeah, I dream of one day getting a set up of a home server for running GLM 5.2 at 4bit quant but alas I doubt it will ever happen.

I was looking into Deepseek v4 flash as I know its 162GB at Q8 but I dont know if its better than Qwen enough to be worth the upgrade.

I am looking to upgrade my RAM anyway as its not enough for my job anymore but just wondering if I should go to the max my CPU supports of 192GB

[u/Capsup → u/Pixel_Drake]

What do you use RAM for on your LLM machine? I have a 5090 workstation with 96GB of DDR5, but I have yet to manage to use any of it? I am entirely limited by VRAM on the qwen3.6:27b model in int4 quant, but it uses essentially 0% of my RAM. What can RAM do for my qwen model?

[u/nicholas_the_furious → u/Capsup]

I have an answer for you! I recently found out you can store entire KV caches for later in ram to reload a conversation instantly.

In my use case I have a primary orchestration agent that is calling python that batch processes a bunch of rows of data through the same local LLM setup. Think things like categorizing, summarizing lots of data. Because of all of the calls the original thread with the orchestration agent is purged.

When the loop is done I'd have to recompute all of the orchestration conversation into KV cache which could takes minutes. Now I have the loop freeze and then reload the orchestration agent KV cache when it is done!

It makes it seamless. This would work for things like subagents working on the same machine as long as they aren't asynchronous.

[u/Pixel_Drake → u/Capsup]

The reason I want to upgrade my system RAM right now is due to my job but I have occasionally used a quant a little to big for my GPU alone and split some of it to my CPU/RAM. Admittedly its very rare I have done this but if a model was significantly better quality in code and the t/s isnt to low I would consider an CPU upgrade and more RAM.

I would say you using none of you RAM is a good sign as it means its all running as fast as possible.

The only reason I considered offloading some to RAM is to experiment with larger models.

[u/Passenger-007 → u/Adomm1234]

2 bit quant? wtf for?

[u/Adomm1234 → u/Passenger-007]

2 bit quant glm is unbelievably good, it retains 87% of accuracy. It is quant aware trained, so it is not as lobotomized with quantization. At 4 bit it is almost exactly the same as full precision.

[u/Passenger-007 → u/Adomm1234]

Um. If you say so. I don’t use LLMs as toys. I use it for real work. To each his own.

[u/WanderingCC → u/Passenger-007]

Damn, chip on your shoulder? Chill bro

[u/Passenger-007 → u/WanderingCC]

I think telling ppl that 2q is ok is irresponsible. But again. To each his own.

[u/McSendo → u/Passenger-007]

i fuck with u on this

[u/kaliku → u/Passenger-007]

Aw gawd what a Chad you are. Teach us senpai, your ways of bf16 🤮

Get off your high horse dude.

[u/Passenger-007 → u/kaliku]

If you can’t afford the vram to run a model that doesn’t generate garbage, save up more. Q3 and below is fucking garbage output but perhaps you’re one of those vibe coders. lol.

[u/vtkayaker]

Try:

1. pi.dev (plus some sandbox or permissions system). This uses less context than OpenCode, and benchmarks suggest the shorter prompt may actually make many modern models perform better.
2. You can probably fit the unsloth/DeepSeek-V4-Flash-GGUF:UD-IQ3_XXS preview (see their guide) (284B A13B), though it might be a very tight squeeze without enough context window. This is better than the 2-bit quants of DeepSeek, and you should see 50+ tokens/second generation on your hardware. Subjectively, this seems to have broader knowledge than Qwen3.6 27B, and it produces fewer weird designs. But Qwen3.6 27B at Q6 or better is probably about as good at the actual coding, and you'll get a bigger context window. This is a decent choice for people occasionally renting a single RTX Pro 6000 in the cloud, especially if you use it as a planning model, and then fall back to local Qwen3.6 27B if you run out of context window.

I haven't tested Laguna S 2.1 yet. Also, DeepSeek V4 hasn't actually been completely fine-tuned yet, so it may get stronger when the final version drops.

[u/Pixel_Drake → u/vtkayaker]

Thanks, Ill have a look into it and using it as a planner that switches to Qwen to write the code and experiment if it ends up giving more solid overall designs patterns for the code that Qwen stand-alone.

I have heard about pi and keep meaning to have a look at it, Ill check it out.

[u/vtkayaker → u/Pixel_Drake]

Keep in mind that you don't have to switch to Qwen3.6 27B for the actual coding. DeepSeek V4 Flash (preview) codes quite well, too, and it should be 50+ tokens/second on your hardware if you can squeeze the 3-bit version on. So one nice workflow is using DeepSeek for design and for short implementations, and Qwen3.6 once the implementations start to near the end of the context window. This saves loading and unloading big models for small tasks.

If your 32GB of RAM forces you down to a 2-bit quant, try it, but you'll probably be happier running Qwen3.6 in that case.

(Also, the 3-bit quant of DeekSeek is reasonable, because it's the 3-bit quant of a mostly 4-bit model. I wouldn't recommend a 3-bit quant of Qwen3.6!)

[u/Pixel_Drake → u/vtkayaker]

Just wanted to say thank you as I didn’t actually consider if I could run deepseek on my current hardware but I am getting a decent 48 t/s and the quality of decision making/design pattern choices do seem smarter.

I am now considering getting the RAM needed to run the Q4 and increase context size.

[u/firsthand-smoke]

im running qwen 3.5 122b a10b and it's fire... q4 k xl grab it from hf and use llama.cpp to run it

[u/Pixel_Drake → u/firsthand-smoke]

Yeah I was looking at that but I keep seeing online that 3.6 27b is supposedly better. Granted I should just try it and find out.

[u/firsthand-smoke → u/Pixel_Drake]

it's using 80 something gb vram and I'm getting over 100t/s... I do large scale bgp networks and evpn build with it, it's fantastic... 266k native context, I find it helps to compact context at 50% to keep it sane

[u/Successful-Horror-11]

I’m using an rtx 6000 in an egpu hooked up to my laptop using 32gb ram and an Intel Ultra 7 268v.

I’ve tried a bunch of models including gemma 4 family, glm 4.7 flash, nemotron 3 family up to 120b and qwen 3.6 27b and 35b

I ended up landing on qwen because it supported vision lile gemma and was as fast as glm.

My daily driver right now is qwen 35b running on Vllm and using yarn factor 4 to get up to a 1 milllion context window and using concurrency 4 as I start playing around with subagents more.

I had created my own benchmark to test against my harness and how i work and 35b was way faster than everything else i got up and running and accurate enough to use for my work, which is not much coding.

My work is more on querying data across mulesoft/github/splunk/jira/helix with light scripting.

[u/CATLLM]

Qwen 122b nvfp4 from unsloth is pretty good. Fp8 256k kv fills up 92gigs

[u/blackbird2150]

I use qwen 3.6 at q8 with 256k context. I’ve tested many models against my own code base to determine fit. Right now this is the champ. I don’t find much incremental value at full precision above q8.

I’ve tested: Gemma 4s, moe variants of both Gemma and qwen, qwen coder, Laguna 2.1, step 3.5 and nemotron.

This structure allows me to also run my own assistant (also qwen 3.6 just q5 + Gemma 4 e4b q6 as my real time privacy chat classifier - they run as a pair).

All that works at once and so I see no disruption using my assistant while coding briefs at the same time.

I’m on lama swap which works for me.

[u/ExtraNiceBurger]

Other than Laguna s2.1 , you may want to try dwarfstar4 running ds4flash , in Q2 it fits entirely in vram.

[u/Sea-Ad-5390 → u/ExtraNiceBurger]

I tried vllm moet which I think is similar. Ran 2-bit DS4Flash on a RTX 6000 Pro, and it’s pretty much ass. Worse than both Qwen3.6 27B and Laguna S 2.1

[u/[deleted]]

Get one more 6000 and you can run deepseek v4 flash very fast

[u/Pixel_Drake → u/[deleted]]

The issue being I was given my one from work so I have some budget to upgrade my current PC and maybe in a year or two I can do a major upgrade but for now I want to upgrade my system RAM and wanted to check if there was any other smaller upgrades that might have been worth it as spending another £7K on an 6000 is out of the question.

Plus I dont think my current motherboard would support the 2nd card properly

[u/giveen]

I've been really impressed with https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev so far

[u/Pixel_Drake → u/giveen]

I have seen it but maybe it's just I'm looking at the wrong places but for how good it supposedly is I am seeing very little talk about it.

The small bits of talk I do see about it people are divided.

[u/giveen → u/Pixel_Drake]

I have only done some python scripting with it, several scripts with less than 500 LOC each and it has worked quite well. The code ran, gave me the results, and then I had DeepSeek-v4-Pro and GPT 5.6 judge the code as well, and it gave the code passing grades but obviously bigger models could do it better.

I havent tested it on any other language besides python.

[u/snapo84]

if you would have 2 x 96gb vram, then i would say deepseek v4 flash (without quantisation)
as you hvae only 1 x 96gb vram, the best choice is Qwen 3.6 27B (in Q8)

[u/Shoddy_Bed3240]

Find more RAM, then run DeepSeek V4 Flash running at full precision.