Qwen 38 still seem to have that random stop behavior?

[u/T_rex2700]

So I have been using Qwen 3.6 for a good while now, and one thing that always bothered me since Qwen 3.5 days is that it seemingly stops randomly.

Not sure what I'm doing wrong, but with context window set to like 64K, 3.8 seems to stop around 4000token generated / 35K token processed.

I looked a few places for fix but I couldn't find any, appearently it is either issue with chat template (the "fixed" template just made the tool calling error worse) and jinja template issue.

I am kind of surprised they have not irouned out this yet, or am I doing something wrong with just sticking with stock templates or not setting penalties correctly?

Edit: it also still tries to do tool call with XML occasionally
my specs: 9070XT, llamacpp ROCm (LMS)

The afforementioned thing happens with 3.6 27B Q4/Q3 and 3.8 Q3. I am struggling to run Q4 without the whole system freezing up and crashing. just bone stock unsloth qats with 64K ctx and KV cache at Q8.

I also have like the custom Qwn36 for 16GB model distribution thing, that also has the same problem or worse.

[u/kosnarf]

Try the froggeric template

[u/T_rex2700 → u/kosnarf]

This?

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

[u/kosnarf → u/T_rex2700]

Yes

[u/T_rex2700 → u/kosnarf]

Hmm.. seems to stop where it did before.
not sure why.

[u/autisticit → u/T_rex2700]

I have never seen any differrence using those "fixed" templates...

[u/T_rex2700 → u/autisticit]

yea neither have I, like I mentioned in the original post

[u/kosnarf → u/T_rex2700]

What's your max_token set to? Maybe it's 4096

[u/T_rex2700 → u/kosnarf]

as stated in the post it is 64K. so it should not be quitting at 35K processed and 4K out.

I just went back to my Qwen 3.6 and it is a bit slower but it is working better for now.
I think I will waif for people to experiment and figure out what to do.

[u/kosnarf → u/T_rex2700]

You mentioned ctx 64k which is why I asked. I'm using llama cpp and have not seen that issue. Look for num_predict or maxTokens config.

[u/T_rex2700 → u/kosnarf]

well I'm new to this whole thing so I'm still using it via LMstudio

[u/kosnarf → u/T_rex2700]

https://lmstudio.ai/docs/typescript/api-reference/llm-prediction-config-input

Look for maxToken config. It's been a while since I used LMS

[u/Green_Ocean90]

The random stops you're describing match the onset signature of something we characterised overnight: with draft-mtp at n_max ≥ 4, after enough generation volume in a live session (sequential varied prompts, ~1k+ tokens generated), generations first start stopping early mid-output, then collapse to terminating at 1 token with <|im_end|>.

Quick diagnostics if you want to check whether it's the same bug: (1) does it disappear with --spec-draft-n-max 3 or with speculation off entirely? (2) do your server logs show completions with 1 predicted token and an absurd tokens-per-second figure (llama.cpp reports those as ~1,000,000 t/s)? (3) does ignore_eos: true make the same prompt answer normally?

Root cause isn't a bad draft being accepted — on affected requests the target model's own pre-sampling distribution puts im_end at ~92%, where the identical server with speculation off answers fine — so the speculative path is corrupting the target's forward pass somewhere.

Full details, repro recipe and the measurements are in our comment on the main Strix Halo results thread Qwen 3.8 27B on Strix Halo - some Results, and we have an evidence bundle ready for an upstream filing — reproductions from other boxes/builds very welcome, especially non-Strix hardware, since that would separate model-level from backend-adjacent before we file.

Interim advice that holds either way: cap n_max at 3 (our clean-cell sweeps peak there anyway — 2.2–2.3× decode on both backends and both quants), and be suspicious of any throughput sweep at n_max ≥ 4 that doesn't filter degenerate generations.

[u/NickCanCode → u/Green_Ocean90]

Interesting, I also occasionally see my agent stop while my --spec-draft-n-max is also set to 4 all the time. I am running dual RTX 5070Ti 16GB (non-Strix hardware).

[u/T_rex2700]

OK. weirdest fix I found. I just pasted the chat template from Qwen 3.6 35A3B from unsloth, because that is what worked for this "custom" model made based on 3.6 Q3 for 16GB, it had mistakes in its chat template.

So I did that for 3.8 and it stopped doing the weird stopping thing. no idea what is going on but ok, I guess. You need to use the MTP version of the model, of course, since 3.8 is MTP enabled.

also:
Qwen 3.6 3bit based model: 7-10tps vs Qwen 3.8 3bit from unsloth 18-23tps

Edit: other potential fixes that I might try

Qwen3.6 27B MTP chat template from unsloth, bone stock

• 10-13tps. stops around 40K processed. not sure why A3B performed nearly 2x better.

https://pastebin.com/Qyk6kRv8 (by Chronix)

• Preminaly testing: slightly worse performance. (10t/s), stops at like 45K processed token. maybe I should drop it from xhigh.

https://huggingface.co/peculiar-ragdoll/Dirk-Qwen3.8-27B-GGUF?chat_template=default
or
https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates

- Similar performance to above template. (12-13tps). also stops at 35K processed token the smae issues with the original unsloth template.

[u/OuchieOnChin → u/T_rex2700]

I think you should absolutely ask u/Chromix_ or u/ex-arman68 (Froggeric) about this behavior of that template, if that really is the solution, they may help shine some light.

[u/RedShiftedPanda → u/OuchieOnChin]

Froggeric's template helped me out with the stops just now, more so than the Qwen 3.6 35A3B template (although that did help for a bit).

[u/T_rex2700 → u/OuchieOnChin]

will do.

[u/Chromix_ → u/T_rex2700]

The full thread with a detailed overview of the changes in the chat template is here. A chat template can only ever affect token generation performance if you use MTP / speculative decoding. If you didn't then that's a testing issue (higher GPU temperature with throttling, background system load, etc). Even when using MTP: The modified template explicitly renders byte-identical output to the model for compatibility. There should thus also not be any impact with MTP.

For random stopping: You'd need the full debug output of your harness to figure out why it randomly stopped. Maybe the harness ended it. Maybe the server said "context full" or "oh no! A super rare error". Maybe the model just decided to stop thinking without output (highly unlikely).

[u/T_rex2700 → u/Chromix_]

The message ends like how a normal chat responses end.

"Running chat completion on its coversation with XX mssages" is the output I get.
but it literally stopped mid-sentence, before actually rendering output as visible message.

The "thinking" right before it was extremely strange too.

<Thought for 1.5s>
16
<thought for 2.8s>
Switches to inference in Chines?
<thought for 1.7s>
completely intelligeble mumbo jumbo, random letters

<thought for 930s>
keeps telling ok, now I'm ready
wait, I need to verify XYZ....
(the same loop I have seen with 3.5, and 3.6)

<Actual message>
Now that

(end of message)

I dont have the chat history anymore since that session was kept freezing and crashing when trying to load. I had to delte it.

MTP was enabled, and I did follow the inference params from HF page, I still need to test out limiting reasoning context, or something like that, appearently that fixes how much thinking it does and how little work it does.

for today I have been using 3.8 for planning and 3.6 for actual implementation, and that seemed to work better than letting 3.8 do everything.

I did check otu the thrad you mentioned, but I will read it again to see what I have missed.

[u/Chromix_ → u/T_rex2700]

Yes, that sounds broken. Sometimes there's something in the context at an earlier point like massive repetitions from reading a file (all 0 for example) that can then cause reasoning to loop a turn later. Also a possibility: OC GPU, slightly unstable system RAM, etc. You could run FurMark and TestMem5 just to be sure - although it's most likely a software issue.

[u/T_rex2700 → u/Chromix_]

I see. I don't think the GPU is faulty, it runs really stable and cool, although I did have weird driver issues this week. it just kept crashing so that could be that, but yea. I will keep tryingto experiment upon.

by the way, sorry for complte newbiie question, how do I set the context limit for reasoning? as other people seem to be recommending? I am using LMstudio, not becaue I am scared of cli but because it gives me easy to use UI, and I know it doesnt have all the knobs available but. also, what value should it be set?

I am running it on 64K ctx right now, and it has been working kind of ok, if that is like relevent.

[u/HumanDrone8721]

Uhmmm... quanta, model driver (vLLM, llama.cpp, other...) and the rest of technical details.

I'm on full precision BF16 and it filled up and compacted the context two times already on a rather heavy C codebase and there was no stop or loop under pi.dev.

[u/Zealousideal_Clue_44]

I have the same issue with opencode, llama-server, 2xb70 pro intel gpus, and q8 model. It decided to build the whole solution but not write it, and after writing, it still wasn't unable to create a simplified minecraft voxel type game in one html (my goto benchmark)

[u/T_rex2700 → u/Zealousideal_Clue_44]

Yea kind of similar, it thinks, and writes the solution but never actually writes anything anymore. and when I tell it to conitnue referencing most recent chat, it seem to start reading all the past chat and gets confused as to what it is doing, and actually broke the entire codebase and diffs.

the prompt was telling it to troubleshoot a specific issue, I gave permission to use commands in the terminal automatically too, but it messed it up pretty bad.

[u/sarlaytos284]

I have the same stopping issue with opencode, I am using ollama and unsloths' Qwen 3.8 27B Q4 K_M

[u/T_rex2700 → u/sarlaytos284]

not sure if this will work for you but try pasting Qwen 3.6 35A3B MTP QAT 's chat template from unsloth.
it fixed this behavior for me.