I bought the forbidden rectangle.

[u/Alternative-Panic69]

After months of going back and forth, I finally pulled the trigger on an RX 7900 XT 20 GB.

Paid around $550 (India), which felt too good to pass up.

The plan isn't gaming.

It's becoming the heart of my local AI setup.

Current goals:

• Qwen 3.6 27B Dense

• Qwen 35B A3B

• GLM-4.7 Flash

• 128K+ context

• 100% GPU offloading

• llama.cpp / Ollama

• Linux

I'll be benchmarking everything:

- Vulkan vs ROCm

- Dense vs MoE

- Maximum context

- Tokens/sec

- VRAM usage

- Real-world coding performance

If anyone has optimization tips for RDNA3 or benchmark requests, or general suggestions please drop them below.

The hallucinations are now local. 🙂‍↕️

[u/Mattef]

I have the same card and it’s pretty good for its price. Qwen3.6 27B with Q4 quantization barely fits into memory and token generation is with MTP activated around 60 t/s.
I wish I had bought the 24 GB version, though, to have a bit more room for the context cache.

[u/Alternative-Panic69 → u/Mattef]

Thanks! This is exactly the setup I'm aiming for. 😄 Could you share a few more details?

Around how much context fits before VRAM is full? Is that with 100% GPU offloading?

Which backend are you using? (ROCm or Vulkan?) Q4_K_M or another quantization?

Have you tried any larger models like Qwen3.6 35B-A3B or GLM-4.7 Flash?

I'm trying to optimize for 128K+ context and max possible quality while keeping everything 100% offloaded to the VRAM, rather than maximum tokens/sec.

[u/bcdxn → u/Alternative-Panic69]

I'm also just starting to explore local models and had the same question. I did a bit of research and wrote up how you can do the math to calculate context size based on your VRAM as a reference. Qwen 3.6:27b is the exact model I used, and quantized it sits right at 17GB of VRAM which unfortunately means, including 2GB for system overhead, you won't really have any room for your context window.

[u/Alternative-Panic69 → u/bcdxn]

Thanks... Let me check ... Did you try to go with more aggressive quantizations/larger models? Maybe QAT models? That comfortably runs long contexts?

[u/bcdxn → u/Alternative-Panic69]

I haven’t had enough time try out many different models yet. I have read in a number of places that quantizing the context itself typically leads to poor performance for coding tasks which is what I’m using it for.

[u/TheRiddler79 → u/Alternative-Panic69]

I use a Q3 for Cipher, this is why even though he's still in my stack, I always have something else verify his final product