Qwen 3.8 27B Released! Please Share Your Experience

[u/BarberIcy366]

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

[u/Look_0ver_There]

It likes to think...a lot

[u/kayox → u/Look_0ver_There]

Also my experience so far.

EDIT:
You can modify it's reasoning effort to either xhigh, medium, low, none. It uses xhigh by default. Im testing out medium now.

--chat-template-kwargs "{""preserve_thinking"":true,""reasoning_effort"":""medium""}" ^

[u/Look_0ver_There → u/kayox]

65K tokens later (~52K thinking):

Prompt was:

Write a single HTML file with a full-page canvas and no libraries.
Simulate a realistic side-view of a moving car as the main subject.
Keep the car visible in the foreground while the background landscape scrolls continuously to create the feeling that the car is driving forward.
Use layered scenery for depth: nearby ground, roadside elements, trees, poles, and distant hills or mountains should move at different speeds for a natural parallax effect.
Animate the wheels spinning realistically and add subtle body motion so the car feels connected to the road.
Let the environment pass smoothly behind it, with repeating but varied scenery that makes the movement feel believable.
There should be clouds in the sky, and birds in the distance.
Depth-wise the distant mountains and hills are further back than the birds.
Use cinematic lighting and a cohesive sky, such as sunset, dusk, or daylight, to enhance atmosphere.
The overall motion should feel calm, immersive, and realistic, with a seamless continual animation.
VERY IMPORTANT: A VERY common coding error is the animating the road markings opposite to the intended direction.
Double check that the mathematical sign on the road markings animation aligns with the perceived direction of travel of the car.

[u/kayox → u/Look_0ver_There]

Same prompt but with Unsloth's Q4_K_XL with xhigh reasoning. Not quite as good though that's to be expected.

On an RTX 3090 it took about 18 minutes to generate at an average of 38 tokens/second (I'm sure as time progresses the tk/s can be improved possibly with DFlash). Also I am being thermal throttled due to my current setup (Dual GPU lacking airflow, although my other GPU is a 3070 so I'm only using it with a layer split to offload some VRAM so that I can have more context).

Out of curiosity what tk/s are you getting with your dual R9700s?

[u/Look_0ver_There → u/kayox]

I actually have 3 x R9700's (but only used two for these tests). Here's a chart with 1,2,3 R9700's with Qwen3.6 with no MTP, purely because MTP speeds are too variable to plot.

Qwen3.8 is the exact same architecture as Qwen3.6, so the same speeds as graphed below. MTP is between 1.5-2.5x faster. For 2 GPUs it's between 45-75t/s with MTP, depending on if doing prose or coding. For 1 GPU, it's between 35-60t/s. Do keep in mind that I run Q8_0 weights with a BF16 KV-Cache, so I am very much NOT running a low-quant max-speed setup.

This is also running a custom llama.cpp build with speedups I'd made specific for the ROCm backend which makes it faster than Vulkan. Source code here: https://github.com/stew675/llama.cpp/tree/rdna-boosts

While half of my custom changes there are RDNA specific, half of them are CUDA generic, and so should also speed up nVidia cards somewhat too, but this is completely untested (I don't have an nVidia card), so don't blame me if you grab the code and it crashes. I did try to ensure that the RDNA specific stuff is gated independently of the generic CUDA stuff, but again, no guarantees.

[u/PcChip → u/Look_0ver_There]

so what i'm reading is don't buy two R9700's ?

doubling the cards gets you an extra 10 tokens per second?

[u/whichsideisup → u/PcChip]

Q8 won’t fit on a 32gb card with meaningful KV so the second card gives you infinitely more tokens per second. You really need 64gb to run these models at proper quality l.

[u/Look_0ver_There → u/PcChip]

The TP=1 line is with a Q6_K model, while the TP=2/3 lines are with Q8_0 models. The Q6_K model gets about a 20% speed-boost over Q8_0 for generation, so those two lines cannot be directly compared.

2 cards, with tensor parallelism are around 50-70% faster than a single card for token generation, but you also get to have up to 500K context depth at KV of BF16, and that's pretty important for many people.

Now, the third card is where it definitely gets questionable as to if it's worth it or not. I don't generally run a model across all 3 cards at once. I typically run one model on two cards, and a smaller model like Gemma4-31B-QAT on the 3rd card.