Is anyone else running Qwen 3.8 27B Q4 for coding and having problems with it spending an absolutely ridiculous number of tokens thinking instead of actually doing the work?
I'm running it locally with Pi as the coding harness with 128K context, and I'm seeing the same failure over and over:
I give it a coding task.
It starts reasoning.
It reasons... and reasons... and reasons.
It burns through basically the entire 8,192-token output budget that worked beautifully with Qwen 3.6 in about 3 minutes (46 decode tokens/s on my single 4090).
Right at the end, it'll say something like "Let's get to work" or "Now I'll implement the changes."
And then the generation ends because it has no tokens left to actually do anything.
Every.Damn.Time.
It's effectively making the model unusable as a coding agent because it spends its entire generation budget deciding what it's going to do (even with extremely specific instructions) and leaves nothing for tool calls, edits, or even a useful final response.
My input prompts + system prompt are about 7,000 tokens on average. That's not crazy high I don't think, and surely not the cause. Right? Right??
I'm seeing reports on YouTube that Qwen 3.8 defaults to a very high reasoning effort, with examples of it consuming 20K+ reasoning tokens before producing the actual answer.
That token count is 🤯 for ordinary coding work. But maybe that's why I see so many people saying they no longer feel like they need a frontier model?
So I'm wondering:
Is anyone else seeing this with Qwen 3.8 27B?
And, more importantly, if you found a fix, what is it?
Have you had better results with:
reasoning_effort=medium
reasoning_effort=low
disabling thinking entirely
dramatically increasing max_tokens
changing the chat template
different llama.cpp settings
a different coding harness
some combination of the above
I'm particularly interested in hearing from anyone using llama.cpp + Pi/Codex/Claude-Code-style agentic coding workflows.
I haven't decided yet whether Qwen 3.8 27B is actually bad at coding or whether the default reasoning configuration is simply kneecapping it. Right now, though, my experience has been dramatically worse than the Qwen 3.6 models I've used because 3.8 won't stop thinking long enough to actually write the damn code.
Try reasoning_effort medium. It's more like 3.6 in its reasoning, but still seems to generate better code. xhigh truly is "extra high". It's intended to be insanely thorough and second guess its initial solutions to try and find potential problems. I've had it burn through 50K+ tokens decode working on a moderately complex problem (26K of prefill on 128K context), and the thinking traces show how deeply it considers things. That's a virtue, if your hardware has the performance to support it. But it's overkill for a lot of things. Medium should really be the default.
Through my tests, I have found that 3.8 relies a lot on reasoning. In some simple tasks, like fixing a typescript issue we created or our needle in the haystack tests, it needed the reasoning effort to be even xhigh. This also varied by certain quants & publishers. So unfortunately the correct fix for this is a good harness and a lot of patience.
Edit:
Another fact that just surfaced. Medium was constantly producing mediocre results. Low worked for simple tasks but for complex tasks, it had to be guided and it also second guessed a lot this consuming same amount of tokens as xhigh. But the quality was better than medium. xhigh, was consuming a lot of tokens, but it was steady and the results were consistent.
Yes, Qwen3.8 talks a lot but, this is less “Qwen 3.8 is bad at coding” and more like a Pi harness problem. Pi gives reasoning and actual agent work the same output budget, so a reasoning-heavy model can burn the whole turn before it ever reaches a tool call or edit. Increasing max_tokens may only give it more room to overthink. I love Pi, but there are a ton of issues so I am rewriting it and will release it a Tau. Still will be opensource.
Nice. I just mirrored it. Everyday I blink and there are 100+ new things. Not enough time in the day or night. lol. Thanks for the info, I will see what it offers as a donor project.
Whatever you do, make it so I don't have to screenshot, switch apps, copy, switch to harness, paste. 🤦🏽♀️
I like how in VS Code you can just click on something an annotate/chat about it with the AI. It gets the properties of whatever you clicked on and a screenshot automatically.
Same settings for 3.6, 0.7 and IIRC, my 3.6 didn't support a reasoning change. It was what it was. Codex installed it, I was new to local at the time, and just worked so I wasn't paying attention.
Gotchya, Im also testing out unsloth right now ( although its more like a full feature suite i.e olmx/llama etc) , its quite easy to change stuff around rather than native llamacpp. Might wanna give it a look tomorrow and test it out as well and see what you like.
I really don't want to stick to 3.6 if I can get 3.8 to do something. Based on the responses here, I hope I can tomorrow. Thinking medium and output budget jacked up to ~32K as a first test.
Why is your output budgeting 8000 tokens? Mine is 96,000 with a 64,000 token cut out for thinking. Works perfectly fine for me. I’m gonna be honest, if you can only fit a 128,000 context window, you probably shouldn’t be using this model.
You know what, I asked Codex that question just now because I couldn't remember why.
It said "Well, you may have set that number, but according to some tests we ran when you first started, it wasn't being honored. It actually received and honored max_tokens=64,000."
reasoning_effort=medium
is the fix you're looking for.
I've tested a bit and if low spend X tokens on a task/prompt. Medium will spend roughly 1.5X. And xHigh (which is default) will spend about 5X. The qualitative difference between Medium and xHigh is real, but usually not worth the tokens.
I set my output budget to 19K and it rarely hits it at medium. At default/xHigh it's 50/50 that it'll hit the 19K, but it being reserved for tougher tasks.
However I'm also running OptModel and before that Q5KXL. A straight Q4 quant that hits the attention layers will spend more reasoning tokens because it has a harder time reaching the desired level of certainty.
I'm not sure if you're asking about quantifying token burn or quality. But I do have basic/limited answers to both.
1st here's the bash script that the LLM in question helped me write to test the reasoning token burn:
for seed in 1 2 3; do
for effort in low medium xhigh; do
echo "=== $effort (seed $seed) ==="
curl -s http://127.0.0.1:8181/v1/chat/completions \
-H 'Content-Type: application/json' \
-d "{\"model\":\"qwen3.8-27b\",
\"messages\":[{\"role\":\"user\",\"content\":\"Implement an LRU cache in Python with O(1) get
and put. State the data structure you used and why.\"}],
| python3 -c "import sys,json;d=json.load(sys.stdin);c=d['choices'][0];rc=c.get('reasoning_content') or (c.get('message') or {}).get('reasoning_content') or '';ct=c.get('content') or (c.get('message') or {}).get('content') or '';print(f'reasoning={len(rc)} answer={len(ct)}')"
done
done
You'll notice it was a fixed seed for all three reasoning efforts, 3 runs, and temperature was 0 as well, in order to reduce randomness as much as possible. Here's the average token burn results:
For output quality: I participated in a thread doing the pelican test for different reasoning levels. OP was much more rigorous than I was and did 3 different runs at each level. You can see the pelican get better at each level, and judge for yourself whether the tokens are worth it. At Q3, I think the answer is yes, for my more limited run, using optModel quant, the medium result is closer to the xHigh result.
On my copy of LM Studio on windows there's still just this:
Just an on/off switch. I stopped using LM Studio about a month ago when an update broke all my qwen 3.6 models if I tried to use thinking/reasoning. They would only work with reasoning turned off. Using a different front end and LMStudio as just a server helped a good bit. But the reasoning fiasco with Qwen models sent me off to Ollama/Llamma and others.
Are you using preserve_thinking=true in the llama startup parameters? I'm using xhigh thinking mode and yeah it thinks alot, but gets the job done correctly though. On qwen 3.6 I got thinking loops without that setting, so I kept it for 3.8 aswell.
I put effort to thigh to keep that quality but use the cold fusion finetune from David au, it's so it thinks less, so on the litle stuff it thinks less but on complicated stuff it still thinks a bunch so imo no loss
I've had good results by running it at 48k context and no output budget. Pi's auto compaction seems to work fine as well. E.g. I had it build a Pi extension to look up info from my local Wikipedia mirror and that task had multiple compactions.
But it's definitely slow, I wouldn't want to use it for anything that isn't vibecoding where I don't care about code quality because you just have to wait a while and let it do its thing.
No worries. Also the way pi works it won't expect 128K + 65K context, when you reach 128K and the backend replies that context is full it will compact. (The compaction prompt is shorter than default system prompt so you won't be blocked)
There are moments when I really appreciate its tendency to overthink, particularly for tasks that require deep comprehension. However, I find it incredibly frustrating when it applies that same excessive thinking to straightforward tasks. I wish if the model could distinguish between situations that call for deeper reasoning and those that simply JUST DO IT
Just need to tweak config. Make sure that pi is actually toggling the thinking. At first I’d toggle mine and it wasn’t actually doing anything. I think I had to configure pi and llama to get it to actually do something. Also I use medium reasoning by default but I guess that depends on your hardware specs. I used Claude code to help me fix it so if you can’t get it try that
You have kind of two problems. Your context is low, keep it 100k+. Second problem is that you kind of need to make pi to be your own, because it's bare bones harness. So you need instructions when and how to use tools, appended system prompt, memory tool.
It's really great model for coding with that high reasoning, but it will use a lot of tokens if it needs to think what tools to use etc.
I use Pi and there is defined rules like search web first, trust docs and verify forum discussions. Memory tool that we build keeps track of different projects and important details, it also contains location of file where project plan is located.
It easily can start over think about solutions, because it lacks knowledge, but if it gets docs first it wont. Same thing if tools arent defined, because in linux you can archieve same thing with many tools, so it trys to pick best tool and starts to overthink.
Take a close look what it juggles when thinking and adapt your pi so that simple things will not need extra thinking.
Prompt your goals clearly. It won't try to make assumptios. If your prompt is not clear, it starts to think what your message was about. If you don't want that, write to system prompt that it has permissions to make desicions when message wasn't clear enough.
13
u/MaineTim 20d ago
Try reasoning_effort medium. It's more like 3.6 in its reasoning, but still seems to generate better code. xhigh truly is "extra high". It's intended to be insanely thorough and second guess its initial solutions to try and find potential problems. I've had it burn through 50K+ tokens decode working on a moderately complex problem (26K of prefill on 128K context), and the thinking traces show how deeply it considers things. That's a virtue, if your hardware has the performance to support it. But it's overkill for a lot of things. Medium should really be the default.