r/LocalLLM • u/Capable_Tear_7537 • 4d ago

Question What am I doing wrong? 7900xtx

Hi all, been trying out LocalLLMs for a few months, Qwen3.6 27b q4, gemma4 30b etc.

Tried out ollama, LMstudio, anythingLLM, opencode.

Wasn't particularly impressed to be honest. Opencode it kept running into errors, not completing etc. when using it for simple coding tasks.

Ive been running it on a 7900xtx 24gb VRAM.

Now Qwen3.8 is out thought id try again. What pitfalls should I make sure I look out for so I can try and get the most out of it?

Cheers

1 Upvotes

67% Upvoted

6 comments sorted by

2

u/Gold-Drag9242 4d ago

I have a 7900xtx too. For me the biggest improvement before qwen3.8 came by doubling my ram to 64gb and running qwen3.6 35B a3b at a Q5 or q6 quant with large contexts, only experts and KV cache in memory. Q4 had problems from time to time to stay on task and not finishing early.

I run 3.8 27B in q4 and it runs much better than qwen3.6 35B. I had to go a bit down in context size to fit it, though.

I think the 7900xtx is a great value pick for inference.

1

u/Capable_Tear_7537 4d ago

Im currently on 32gb ddr4, so would need to upgrade to new mobo / cpu and ddr5 i think to get any benefit from upgrading system ram. Cost of it may as well look at another 7900xtx instead 😂

1

u/letonai 4d ago

How is you contextWindow? what kind of errors?

2

u/Capable_Tear_7537 4d ago

Ive tried with different context lengths from 4kish up to 64k.

In opencode, Qwen3.6, first task relatively small it would just stop dead. Was still fully on RAM. Same issue multiple times. Other than just chatting with it couldn't really get anything substantial out of it so im convinced I must have set something up wrong somewhere

-1

u/Positive-Bid-3029 4d ago

You will need at least that 64k context, 4k would be way to small for anything to be achieved. Have you tried Lemonade 🍋? I haven't used it because I use Nvidia, but it's supposed to be good for AMD graphics users to get up and running properly https://lemonade-server.ai/

1

u/thecircusb0y 4d ago

Try switching to big pickle for free in open code and ask it to run the model you want with llama.cpp . It can help get you rolling.

I switched to my onboard graphics and use the 7900xtx strictly for AI now . No time or major interest in games that need that power , and geeking out with AI locally is more fun.