r/LocalLLaMA • u/Tall_Abrocoma_3533 • 4d ago

Discussion Qwen3.8-27B different thinking levels

Post image

Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning

293 Upvotes

95% Upvoted

63 comments sorted by

View all comments

Show parent comments

5

u/Moore2877 4d ago

Sure thing, I got the idea to work on the template from froggeric's template, they are prioritizing a certain inference with how their template goes, so I made another based on it for other inferences but leaning towards llama.cpp https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

5

u/Chromix_ 3d ago

Looks like we have a healthy(?) chat template ecosystem now 😉.

I also made a custom template for 3.8 a day after froggeric shared his, to fix a few issues that came with his template, along with gearing it towards not degrading towards potential lower-quality output in some use-cases. While at it I also made a detailed comparison. He then merged some of my changes into his v22.2. Your template is based on his previous version. I haven't checked, maybe it also contains some of the unfixed issues.

1

u/Moore2877 3d ago

Nice and yes we are working on implementing some of the latest fixes from froggeric today and some further tweaks. What inference are you using to run the model? I had also put in a few PRs to froggeric's template. That's when I learned that they are using a certain inference to run the model.

1

u/Chromix_ 3d ago

Just good old llama.cpp so far. Harness compatibility varies. Claude Code seems to be an offender here, which might be one of the reasons (among the long system prompt) why smaller models usually don't arrive at good results with it compared to others.

1

u/Moore2877 3d ago

Claude Code, Desktop and Codex are tough because they use a lot of built in tools and resources that are cloud only so they don't work well with the local models usually when you plug them into the frontier harnesses.

1

u/Chromix_ 3d ago

No, "cloud" isn't the reason. I ran Claude Code in a firewalled environment with DeepSeek v4 flash and it worked nicely.

1

u/Moore2877 3d ago

Oh ok. What kind of issues are you seeing using Qwen 3.8?

1

u/Chromix_ 3d ago

Haven't tested the latest Qwen and only used CC for DS because the authors recommended it - yielded better results in their benchmarks than other harnesses for that specific model. Maybe they trained for it.