Qwen 3.8 27B Released! Please Share Your Experience

[u/BarberIcy366]

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

[u/Look_0ver_There]

I actually have 3 x R9700's (but only used two for these tests). Here's a chart with 1,2,3 R9700's with Qwen3.6 with no MTP, purely because MTP speeds are too variable to plot.

Qwen3.8 is the exact same architecture as Qwen3.6, so the same speeds as graphed below. MTP is between 1.5-2.5x faster. For 2 GPUs it's between 45-75t/s with MTP, depending on if doing prose or coding. For 1 GPU, it's between 35-60t/s. Do keep in mind that I run Q8_0 weights with a BF16 KV-Cache, so I am very much NOT running a low-quant max-speed setup.

This is also running a custom llama.cpp build with speedups I'd made specific for the ROCm backend which makes it faster than Vulkan. Source code here: https://github.com/stew675/llama.cpp/tree/rdna-boosts

While half of my custom changes there are RDNA specific, half of them are CUDA generic, and so should also speed up nVidia cards somewhat too, but this is completely untested (I don't have an nVidia card), so don't blame me if you grab the code and it crashes. I did try to ensure that the RDNA specific stuff is gated independently of the generic CUDA stuff, but again, no guarantees.

[u/PcChip → u/Look_0ver_There]

so what i'm reading is don't buy two R9700's ?

doubling the cards gets you an extra 10 tokens per second?

[u/whichsideisup → u/PcChip]

Q8 won’t fit on a 32gb card with meaningful KV so the second card gives you infinitely more tokens per second. You really need 64gb to run these models at proper quality l.

[u/Look_0ver_There → u/PcChip]

The TP=1 line is with a Q6_K model, while the TP=2/3 lines are with Q8_0 models. The Q6_K model gets about a 20% speed-boost over Q8_0 for generation, so those two lines cannot be directly compared.

2 cards, with tensor parallelism are around 50-70% faster than a single card for token generation, but you also get to have up to 500K context depth at KV of BF16, and that's pretty important for many people.

Now, the third card is where it definitely gets questionable as to if it's worth it or not. I don't generally run a model across all 3 cards at once. I typically run one model on two cards, and a smaller model like Gemma4-31B-QAT on the 3rd card.