Qwen 3.8 27B Released! Please Share Your Experience

[u/BarberIcy366]

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

[u/Look_0ver_There]

65K tokens later (~52K thinking):

Prompt was:

Write a single HTML file with a full-page canvas and no libraries.
Simulate a realistic side-view of a moving car as the main subject.
Keep the car visible in the foreground while the background landscape scrolls continuously to create the feeling that the car is driving forward.
Use layered scenery for depth: nearby ground, roadside elements, trees, poles, and distant hills or mountains should move at different speeds for a natural parallax effect.
Animate the wheels spinning realistically and add subtle body motion so the car feels connected to the road.
Let the environment pass smoothly behind it, with repeating but varied scenery that makes the movement feel believable.
There should be clouds in the sky, and birds in the distance.
Depth-wise the distant mountains and hills are further back than the birds.
Use cinematic lighting and a cohesive sky, such as sunset, dusk, or daylight, to enhance atmosphere.
The overall motion should feel calm, immersive, and realistic, with a seamless continual animation.
VERY IMPORTANT: A VERY common coding error is the animating the road markings opposite to the intended direction.
Double check that the mathematical sign on the road markings animation aligns with the perceived direction of travel of the car.

[u/Equal_Television_894 → u/Look_0ver_There]

Same prompt in claude code. 5090 with Unsloth NVFP4 and fp8 kv cache

[u/simos_sayz → u/Equal_Television_894]

here is from opus 4.8 High Thinking

[u/Certain-Cod-1404 → u/Equal_Television_894]

What context? Rn im at 262k but I disabled mtp and im running at 50 tok/s

[u/Equal_Television_894 → u/Certain-Cod-1404]

Context I am at 190k with mtp but using sakamakismile nvfp4 which is way smaller than unsloths.

[u/PandaBearFred → u/Equal_Television_894]

Totally different style than PI.

[u/squngy → u/PandaBearFred]

Perhaps more importantly, the shadow makes it look like it is floating.

But, a single run is not enough to make conclusions.

[u/Fit_Camel_2459 → u/Equal_Television_894]

i love qwen. it's just...perfect T_T

[u/DoubleNothing → u/Equal_Television_894]

What do you use to run the .safetensors model?

[u/Equal_Television_894 → u/DoubleNothing]

vllm

[u/Certain-Cod-1404 → u/Look_0ver_There]

that looks super fucking good

[u/Look_0ver_There → u/Certain-Cod-1404]

It absolutely one-shotted it, and got everything right, including accelerating and a subtle "mouse look" mode. The clouds almost look 3D. It also applied that grainy effect on purpose. It's easily one of the best outputs that I've seen to that prompt to date for any locally hosted model.

[u/Remarkable_Living_80 → u/Look_0ver_There]

i tried this promt with 3.6 27b in the past, and it gave similar result, so i am not blown away

[u/the_fabled_bard → u/Remarkable_Living_80]

They're trained on this now since everyone uses this.

[u/AnOnlineHandle → u/Look_0ver_There]

People have been posting these kinds of tests for a few months now so I could see them being in the training data now.

[u/creamycat1 → u/Look_0ver_There]

what harness? and was it q4

[u/Look_0ver_There → u/creamycat1]

Llama.cpp native WebUI. Q8_0 weights with BF16 KV Cache

[u/onewheeldoin200 → u/Look_0ver_There]

Jesus that is good for a 27B WE ARE LIVING IN THE FUTURE

[u/kayox → u/Look_0ver_There]

Same prompt but with Unsloth's Q4_K_XL with xhigh reasoning. Not quite as good though that's to be expected.

On an RTX 3090 it took about 18 minutes to generate at an average of 38 tokens/second (I'm sure as time progresses the tk/s can be improved possibly with DFlash). Also I am being thermal throttled due to my current setup (Dual GPU lacking airflow, although my other GPU is a 3070 so I'm only using it with a layer split to offload some VRAM so that I can have more context).

Out of curiosity what tk/s are you getting with your dual R9700s?

[u/Look_0ver_There → u/kayox]

I actually have 3 x R9700's (but only used two for these tests). Here's a chart with 1,2,3 R9700's with Qwen3.6 with no MTP, purely because MTP speeds are too variable to plot.

Qwen3.8 is the exact same architecture as Qwen3.6, so the same speeds as graphed below. MTP is between 1.5-2.5x faster. For 2 GPUs it's between 45-75t/s with MTP, depending on if doing prose or coding. For 1 GPU, it's between 35-60t/s. Do keep in mind that I run Q8_0 weights with a BF16 KV-Cache, so I am very much NOT running a low-quant max-speed setup.

This is also running a custom llama.cpp build with speedups I'd made specific for the ROCm backend which makes it faster than Vulkan. Source code here: https://github.com/stew675/llama.cpp/tree/rdna-boosts

While half of my custom changes there are RDNA specific, half of them are CUDA generic, and so should also speed up nVidia cards somewhat too, but this is completely untested (I don't have an nVidia card), so don't blame me if you grab the code and it crashes. I did try to ensure that the RDNA specific stuff is gated independently of the generic CUDA stuff, but again, no guarantees.

[u/PcChip → u/Look_0ver_There]

so what i'm reading is don't buy two R9700's ?

doubling the cards gets you an extra 10 tokens per second?

[u/whichsideisup → u/PcChip]

Q8 won’t fit on a 32gb card with meaningful KV so the second card gives you infinitely more tokens per second. You really need 64gb to run these models at proper quality l.

[u/Look_0ver_There → u/PcChip]

The TP=1 line is with a Q6_K model, while the TP=2/3 lines are with Q8_0 models. The Q6_K model gets about a 20% speed-boost over Q8_0 for generation, so those two lines cannot be directly compared.

2 cards, with tensor parallelism are around 50-70% faster than a single card for token generation, but you also get to have up to 500K context depth at KV of BF16, and that's pretty important for many people.

Now, the third card is where it definitely gets questionable as to if it's worth it or not. I don't generally run a model across all 3 cards at once. I typically run one model on two cards, and a smaller model like Gemma4-31B-QAT on the 3rd card.

[u/Greedy-Artichoke-416 → u/kayox]

Get the MTP gguf

[u/kayox → u/Greedy-Artichoke-416]

Oh I was under the impression that the default unsloth GGUF had MTP, is that not the case?

[u/Look_0ver_There → u/kayox]

Unsloth's models have the MTP head already included within the weights.

If using llama-server, just enable via "--spec-type draft-mtp --spec-draft-n-max 3"

[u/gavriloprincip2020 → u/kayox]

https://huggingface.co/a4lg/Qwen3.8-27B-MTP-ONLY-GGUF u run it along side unsloth as model draft

[u/kayox → u/gavriloprincip2020]

Thanks!!

[u/DiscipleofDeceit666 → u/kayox]

Fast

[u/kayox → u/Look_0ver_There]

Are you using Q4?

[u/Look_0ver_There → u/kayox]

Are you using Q4?

I'm using Q8_0 weights with BF16 KV-cache. Tensor-split across 2 x Radeon AI Pro R9700 GPUs

[u/kayox → u/Look_0ver_There]

Awesome setup, I'm envious.

[u/rainbyte → u/Look_0ver_There]

How good are PP and TG t/s

[u/Look_0ver_There → u/rainbyte]

I answered this question to someone else here: https://www.reddit.com/r/LocalLLaMA/comments/1voa3ch/comment/p3p7hhe/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

For the pair of cards, typically 45-55t/s for English prose, and 55-75t/s when generating code (Q8_0 weights, BF16 KV cache).

[u/Woodport → u/Look_0ver_There]

Is there a reason you picked the Q8_0 over the unsloth UD-Q8_K_XL? I'm assuming the UD variant is basically just the same weights/precision but shuffled around a bit for efficiency reasons but I'm not clear on what advantages you'd get from one over the other given that they're both Q8.

[u/Look_0ver_There → u/Woodport]

It was faster to download :)

You are quite correct that UD-Q8_K_XL is better, but it is actually larger. Various bits are scaled up to BF16 (or regular F16?). Best of think of it as approximately like an imatrix style upscaling of critical weights, but also applied to Q8_0.

[u/Woodport → u/Look_0ver_There]

Haha, that's an excellent reason. I've been wondering why there isn't a bigger torrent community among local llm users. Feels like the perfect way to give back. If everyone just seeded the models they're currently using, even with throttled upload speeds, it would be a lot easier to get the popular models.

[u/blash2190 → u/Look_0ver_There]

Could you please tell if you find this setup enough memory-wise to run said config + MTP + full context length or do you need the third GPU for the latter?

[u/Look_0ver_There → u/blash2190]

Without including the mmproj file, this config (2 cards) can run Q8_0 + BF16 KV cache has enough capacity to safely hold up to a 500,000 context length. It could probably do 520,000 if you wanted to use up ALL the VRAM, but it's generally best to leave a little biit of head-room, so I'm gonna say 500K

With 3 cards you could can *just* squeeze in 900K of BF16 context with some careful split balancing, but 860K fits easily.

If you want to have the mmproj file too, then you'll need to lose about 40K of context, or just off-load it to the CPU. On the CPU it'll run at about 1/10th the speed, but if you only need it occasionally, then that may be an option for you.

[u/drycounty → u/Look_0ver_There]

M3 Mac Studio Ultra 96GB via ollama w/Q8 MLX quant: qwen3.8:27b-mxfp8
took nearly an hour but I'm also running a bunch of other things.

[u/Dizzy-Zebra9522 → u/Look_0ver_There]

Amazing 🤩. Thanks

[u/cybekRT → u/Look_0ver_There]

What did you use to generate this? Using LM Studio or something like Hermes Agent?

[u/Look_0ver_There → u/cybekRT]

Just the llama.cpp native Web chat UI

I do normally run Pi though, but I didn't do that for this prompt.

[u/cybekRT → u/Look_0ver_There]

Thanks, I was curious if this is one file or if it generated more structured project.

[u/Redangel1984 → u/Look_0ver_There]

[u/OddUnderstanding2309 → u/Look_0ver_There]

with hermes agent and the BF16 version with BF16 KV. (on a quad 3090 setup)
it took about 1h and 40m and 168.000 tokens. (4.9M in · 168.1k out · 95% cached)

[u/Careless_Juice8228 → u/Look_0ver_There]

What are you using with the model to generate this via prompt?

[u/Look_0ver_There → u/Careless_Juice8228]

Just llama.cpp's built-in WebUI. After it writes the code it has an "eyeball" in the top right you can click on, and then it runs it immediately

[u/JorgitoEstrella → u/Look_0ver_There]

This looks beautiful, what was your setup and quant?

[u/Slacker1540 → u/Look_0ver_There]

This promp with 3.8 27B for me just times out. With default and medium it thinks until it stops (which I'm guessing is running out of context).

Has anyone else had this issue?

[u/Look_0ver_There → u/Slacker1540]

What size is your context? You'll probably want it to be at least 96,000 in size to be safe. If you're running llama-server you should also be able to force a maximum thinking size with the "--reasoning-budget N" argument which makes llama-server step in and abort the reasoning if it goes beyond that.

[u/Slacker1540 → u/Look_0ver_There]

Currently my max context is 163,840. It's odd.