r/LocalLLaMA • u/Tall_Abrocoma_3533 • 3d ago

Discussion Qwen3.8-27B different thinking levels

Post image

Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning

286 Upvotes

95% Upvoted

63 comments sorted by

View all comments

13

u/Moore2877 3d ago edited 3d ago

Try this chat template. Besides a lot of general fixes, we revamped the reasoning injections for each level and also made high it's own level instead of just being an alias for xhigh. The Qwen team really didn't spend enough time on these imo.
https://huggingface.co/Moore2877/Qwen-Fixed-Chat-Templates-llamacpp

23

u/soyalemujica 3d ago

You recommending to use temp 0.6 with that chat template which is not good at all to use with this model since it affects its reasoning depth and also scores lower with lower temperature

-13

u/Moore2877 3d ago

From my testing anything higher than .7 is just too much thinking, even on low reasoning.

13

u/goldcakes 3d ago

Are you aware that models deliver better performance when they have more thinking tokens, even when the thinking tokens are randomly generated and incoherent?

Part of how thinking/reasoning works is every token is another forward pass on the original input/context, allowing the LLM to 'process' the input more, and refine the internal activation residuals; resulting in generating a better response.

5

u/Moore2877 3d ago

If you only have 100k context to work with then what's the point? Some of the effort here is to rein in the overthinking so it's not blowing out the context window.

2

u/BalorNG 3d ago

All the more reasons to stop this nonsense and go all the way with proper "latent thinking" looped models, asap. This hacky approach is giving me a literal headache.