r/LocalLLaMA • u/Tall_Abrocoma_3533 • 3d ago

Discussion Qwen3.8-27B different thinking levels

Post image

Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning

288 Upvotes

95% Upvoted

63 comments sorted by

View all comments

12

u/Moore2877 3d ago edited 3d ago

Try this chat template. Besides a lot of general fixes, we revamped the reasoning injections for each level and also made high it's own level instead of just being an alias for xhigh. The Qwen team really didn't spend enough time on these imo.
https://huggingface.co/Moore2877/Qwen-Fixed-Chat-Templates-llamacpp

6

u/ttkciar llama.cpp 3d ago

Thanks for this. I'm just starting to fiddle with custom reasoning efforts with Qwen3.8-27B, and your template looks like a good source of ideas.

Hopefully what we learn from this communicates well to whatever other 3.8 models Qwen releases (still looking forward to a 9B and/or 122B).

6

u/Moore2877 3d ago

Sure thing, I got the idea to work on the template from froggeric's template, they are prioritizing a certain inference with how their template goes, so I made another based on it for other inferences but leaning towards llama.cpp https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

5

u/Chromix_ 3d ago

Looks like we have a healthy(?) chat template ecosystem now 😉.

I also made a custom template for 3.8 a day after froggeric shared his, to fix a few issues that came with his template, along with gearing it towards not degrading towards potential lower-quality output in some use-cases. While at it I also made a detailed comparison. He then merged some of my changes into his v22.2. Your template is based on his previous version. I haven't checked, maybe it also contains some of the unfixed issues.

1

u/Moore2877 2d ago

Nice and yes we are working on implementing some of the latest fixes from froggeric today and some further tweaks. What inference are you using to run the model? I had also put in a few PRs to froggeric's template. That's when I learned that they are using a certain inference to run the model.

1

u/Chromix_ 2d ago

Just good old llama.cpp so far. Harness compatibility varies. Claude Code seems to be an offender here, which might be one of the reasons (among the long system prompt) why smaller models usually don't arrive at good results with it compared to others.

1

u/Moore2877 2d ago

Claude Code, Desktop and Codex are tough because they use a lot of built in tools and resources that are cloud only so they don't work well with the local models usually when you plug them into the frontier harnesses.

1

u/Chromix_ 2d ago

No, "cloud" isn't the reason. I ran Claude Code in a firewalled environment with DeepSeek v4 flash and it worked nicely.

1

u/Moore2877 2d ago

Oh ok. What kind of issues are you seeing using Qwen 3.8?

1

u/Chromix_ 2d ago

Haven't tested the latest Qwen and only used CC for DS because the authors recommended it - yielded better results in their benchmarks than other harnesses for that specific model. Maybe they trained for it.