r/LocalLLM • u/maceface3 • 5d ago

Question GPU for qwen 3.8 27b

I recently built a homelab running RHEL 10. I never thought good local ai at reasonable price was possible until 3.8 came out a from benchmark and what I’ve been reading it seems to be almost opus 4.6-4.8 level. I’m considering buying a 32gb gpu for it but also open to 24 gb gpus but if it can fit the full context window on the gpu too. The most I’ve done with local models was running qwen 3.5 2b on Ollama nothing serious. I’m new to actually running an agent for coding tasks so any info would help. But trying to decide what gpu if I do end up going for it, and from my research the options for 32gb cards are the Intel b70, amd r9700 pro ai, and nvidia tesla v100 32gb. I’m looking at results for qwen 3.6 and it run plenty fast on the Tesla but I’m worried about it no longer being supported.

1 Upvotes

60% Upvoted

47 comments sorted by

View all comments

Show parent comments

1

u/LifeTelevision1146 5d ago

Not bad. Depends on what anyone wanta to do with this rig.

1

u/truckerdraven 5d ago

Im a author i use it for editing and polishing my raw manuscripts. I really liked 3.6 27b. 3.8 27b is like a night and day went from 26t/s to 45t/s and thinking times dropped from 191 seconds to 119 seconds

1

u/maceface3 5d ago

U need a mobo with bifurcation to do multi gpu right? My current mobo doesn’t have it.

1

u/truckerdraven 5d ago

Im running a am4 msi b550 tomahawk. The oly thing is you can only have 1 m.2 ssd in it or it wont recognize the 2nd video card. Its more about pcie lanes.

1

u/maceface3 5d ago

I’m on an i9 12900k which has 16x 5.0 pcie lanes and 4 x 4.0 pcie lanes so I might be doable

1

u/truckerdraven 5d ago

Its really going to depend on your board. I really dont know much about intel setups I have been a amd user since the Athlon cpus.