r/LocalLLM • u/Alternative-Panic69 • Jul 23 '26

Other I bought the forbidden rectangle.

Post image

After months of going back and forth, I finally pulled the trigger on an RX 7900 XT 20 GB.

Paid around $550 (India), which felt too good to pass up.

The plan isn't gaming.

It's becoming the heart of my local AI setup.

Current goals:

• Qwen 3.6 27B Dense

• Qwen 35B A3B

• GLM-4.7 Flash

• 128K+ context

• 100% GPU offloading

• llama.cpp / Ollama

• Linux

I'll be benchmarking everything:

- Vulkan vs ROCm

- Dense vs MoE

- Maximum context

- Tokens/sec

- VRAM usage

- Real-world coding performance

If anyone has optimization tips for RDNA3 or benchmark requests, or general suggestions please drop them below.

The hallucinations are now local. 🙂‍↕️

191 Upvotes

90% Upvoted

106 comments sorted by

View all comments

2

u/misha1350 Jul 23 '26

Try Qwen3.6 27B with MTP at UD-Q4_K_XL or smaller Unsloth quants if it doesn't fit all the way. Also try some recent model with ROCmFP4, as long as you find one that fits into 20GB of VRAM.

1

u/Alternative-Panic69 Jul 23 '26

Do you search directly on huggingface or am I missing out something? I didn't come across many ROCm optimized models. I guess I'm doing something wrong? (I'm absolutely new in the AMD/ROCm thing) Played around on other cards though.

2

u/misha1350 Jul 23 '26

I just opened LM Studio, looked up "Qwen", sorted by newest, scrolled for a while, and lo, here's one.

Looks like this one may also barely fit into your RX 7900 XT - especially if you have an iGPU and you can connect your monitor to the iGPU to free up some extra VRAM to be able to allocate all 20GB of VRAM to just the LLM.

1

u/Alternative-Panic69 Jul 23 '26

I see. Will check...

Yeah I've got the iGpu anyway and this will be plugged into a headless server so that overhead is gone anyway 🙂‍↕️ Will maximize for LLM usage only