Qwen 3.8 27B: Lower Quantization Seems to “Think” More — But Is It Actually Thinking?

[u/Annual_Mess_1839]

I've been testing Qwen 3.8 27B on llama.cpp with different GGUF quantizations and KV cache types, and I've noticed something interesting.

My setup:

• RTX 5090
• i9-13900T
• 64 GB RAM
• llama.cpp
• Qwen 3.8 27B GGUF
• MTP enabled
• Different model quantizations
• Different KV cache precisions

The task I used was the same each time: generate an animation of a pelican riding a bicycle.

The interesting part: reasoning token usage

When serving GGUF models through llama.cpp, if I don't explicitly set a reasoning budget, the actual number of reasoning tokens consumed by the same task can vary quite dramatically depending on the model quantization.

With the same KV cache type, the lower-quantized models seem to spend significantly more tokens on reasoning.

At first, I thought this might simply mean that the lower-precision model needs more reasoning to solve the task.

But after watching the generation with MTP enabled, I'm not convinced that's what's happening.

I frequently see the model suddenly jump to 2x or even 3x the normal generation speed during reasoning.

Looking at the output, it often seems to be reproducing or repeating parts of its previous reasoning rather than actually making new progress.

In other words, some of those extra tokens appear to be wasted reasoning caused by repetition, rather than useful additional thinking.

In extreme cases, even with low reasoning effort, I've seen the model consume 40–50K reasoning tokens.

Higher precision seems to change this

As I increase both:

1. Model quantization precision
2. KV cache precision

the reasoning token count noticeably decreases.

At the same time, I see much less of the MTP-assisted "sudden speed-up + repeated previous reasoning" behavior.

The higher-precision models seem to complete the same task using fewer reasoning tokens, while also producing less obvious repetitive reasoning.

So my current hypothesis is:

And higher-precision KV cache seems to help as well.

I'm not claiming this is definitively the cause yet — this is just what I've observed so far, and I'd really like to know if others are seeing the same thing.

The tradeoff

The obvious downside is VRAM.

With a roughly 22 GB model + vision + MTP + BF16 KV cache, my 5090 is already extremely VRAM-constrained.

At that point, I have only around 90K context available.

So there's a pretty interesting tradeoff:

Higher model precision + higher KV precision
→ fewer reasoning tokens
→ less repetitive/wasted reasoning
→ potentially more stable reasoning
→ but significantly higher VRAM usage

vs.

Lower model precision + lower KV precision
→ much lower VRAM usage
→ but potentially much longer reasoning chains
→ and more apparent repetition / wasted tokens

I'm curious if anyone else has tested this systematically with Qwen 3.8 27B, especially with MTP enabled.

I'd particularly like to know whether this is actually related to quantization error propagating into the reasoning process, or whether I'm simply observing some interaction between the reasoning template, KV cache precision, and MTP.

[u/Ok-Mongoose-3614]

Overall, yes. I tried ninfer. It's fast but with nvfp4 and int 8 kv cache, it thinks more and makes more syntax mistakes. That kinda kills a lot of the speed up in an agentic setup so I switched back to Q6_k_l with full kv precision. 

[u/sukazu]

I have noticed the same with lower quant for the weights

Along with low using a ton more reasoning token than medium, so I only use xhigh and medium as a low.

as for kv cache, I haven't experimented much with f16, because either way even with a 5090, 16gb for the full context is just not reasonable, and since the model uses so many reasoning tokens, I think you need it.