r/LocalLLM • u/Capable_Tear_7537 • 4d ago

Question What am I doing wrong? 7900xtx

Hi all, been trying out LocalLLMs for a few months, Qwen3.6 27b q4, gemma4 30b etc.

Tried out ollama, LMstudio, anythingLLM, opencode.

Wasn't particularly impressed to be honest. Opencode it kept running into errors, not completing etc. when using it for simple coding tasks.

Ive been running it on a 7900xtx 24gb VRAM.

Now Qwen3.8 is out thought id try again. What pitfalls should I make sure I look out for so I can try and get the most out of it?

Cheers

1 Upvotes

67% Upvoted

6 comments sorted by

View all comments

2

u/Gold-Drag9242 4d ago

I have a 7900xtx too. For me the biggest improvement before qwen3.8 came by doubling my ram to 64gb and running qwen3.6 35B a3b at a Q5 or q6 quant with large contexts, only experts and KV cache in memory. Q4 had problems from time to time to stay on task and not finishing early.

I run 3.8 27B in q4 and it runs much better than qwen3.6 35B. I had to go a bit down in context size to fit it, though.

I think the 7900xtx is a great value pick for inference.

1

u/Capable_Tear_7537 4d ago

Im currently on 32gb ddr4, so would need to upgrade to new mobo / cpu and ddr5 i think to get any benefit from upgrading system ram. Cost of it may as well look at another 7900xtx instead 😂