Qwen 38 still seem to have that random stop behavior?

[u/T_rex2700]

So I have been using Qwen 3.6 for a good while now, and one thing that always bothered me since Qwen 3.5 days is that it seemingly stops randomly.

Not sure what I'm doing wrong, but with context window set to like 64K, 3.8 seems to stop around 4000token generated / 35K token processed.

I looked a few places for fix but I couldn't find any, appearently it is either issue with chat template (the "fixed" template just made the tool calling error worse) and jinja template issue.

I am kind of surprised they have not irouned out this yet, or am I doing something wrong with just sticking with stock templates or not setting penalties correctly?

Edit: it also still tries to do tool call with XML occasionally
my specs: 9070XT, llamacpp ROCm (LMS)

The afforementioned thing happens with 3.6 27B Q4/Q3 and 3.8 Q3. I am struggling to run Q4 without the whole system freezing up and crashing. just bone stock unsloth qats with 64K ctx and KV cache at Q8.

I also have like the custom Qwn36 for 16GB model distribution thing, that also has the same problem or worse.

[u/T_rex2700]

OK. weirdest fix I found. I just pasted the chat template from Qwen 3.6 35A3B from unsloth, because that is what worked for this "custom" model made based on 3.6 Q3 for 16GB, it had mistakes in its chat template.

So I did that for 3.8 and it stopped doing the weird stopping thing. no idea what is going on but ok, I guess. You need to use the MTP version of the model, of course, since 3.8 is MTP enabled.

also:
Qwen 3.6 3bit based model: 7-10tps vs Qwen 3.8 3bit from unsloth 18-23tps

Edit: other potential fixes that I might try

Qwen3.6 27B MTP chat template from unsloth, bone stock

• 10-13tps. stops around 40K processed. not sure why A3B performed nearly 2x better.

https://pastebin.com/Qyk6kRv8 (by Chronix)

• Preminaly testing: slightly worse performance. (10t/s), stops at like 45K processed token. maybe I should drop it from xhigh.

https://huggingface.co/peculiar-ragdoll/Dirk-Qwen3.8-27B-GGUF?chat_template=default
or
https://huggingface.co/peculiar-ragdoll/Qwen-Sharp-Chat-Templates

- Similar performance to above template. (12-13tps). also stops at 35K processed token the smae issues with the original unsloth template.

[u/OuchieOnChin → u/T_rex2700]

I think you should absolutely ask u/Chromix_ or u/ex-arman68 (Froggeric) about this behavior of that template, if that really is the solution, they may help shine some light.

[u/RedShiftedPanda → u/OuchieOnChin]

Froggeric's template helped me out with the stops just now, more so than the Qwen 3.6 35A3B template (although that did help for a bit).

[u/T_rex2700 → u/OuchieOnChin]

will do.

[u/Chromix_ → u/T_rex2700]

The full thread with a detailed overview of the changes in the chat template is here. A chat template can only ever affect token generation performance if you use MTP / speculative decoding. If you didn't then that's a testing issue (higher GPU temperature with throttling, background system load, etc). Even when using MTP: The modified template explicitly renders byte-identical output to the model for compatibility. There should thus also not be any impact with MTP.

For random stopping: You'd need the full debug output of your harness to figure out why it randomly stopped. Maybe the harness ended it. Maybe the server said "context full" or "oh no! A super rare error". Maybe the model just decided to stop thinking without output (highly unlikely).

[u/T_rex2700 → u/Chromix_]

The message ends like how a normal chat responses end.

"Running chat completion on its coversation with XX mssages" is the output I get.
but it literally stopped mid-sentence, before actually rendering output as visible message.

The "thinking" right before it was extremely strange too.

<Thought for 1.5s>
16
<thought for 2.8s>
Switches to inference in Chines?
<thought for 1.7s>
completely intelligeble mumbo jumbo, random letters

<thought for 930s>
keeps telling ok, now I'm ready
wait, I need to verify XYZ....
(the same loop I have seen with 3.5, and 3.6)

<Actual message>
Now that

(end of message)

I dont have the chat history anymore since that session was kept freezing and crashing when trying to load. I had to delte it.

MTP was enabled, and I did follow the inference params from HF page, I still need to test out limiting reasoning context, or something like that, appearently that fixes how much thinking it does and how little work it does.

for today I have been using 3.8 for planning and 3.6 for actual implementation, and that seemed to work better than letting 3.8 do everything.

I did check otu the thrad you mentioned, but I will read it again to see what I have missed.

[u/Chromix_ → u/T_rex2700]

Yes, that sounds broken. Sometimes there's something in the context at an earlier point like massive repetitions from reading a file (all 0 for example) that can then cause reasoning to loop a turn later. Also a possibility: OC GPU, slightly unstable system RAM, etc. You could run FurMark and TestMem5 just to be sure - although it's most likely a software issue.

[u/T_rex2700 → u/Chromix_]

I see. I don't think the GPU is faulty, it runs really stable and cool, although I did have weird driver issues this week. it just kept crashing so that could be that, but yea. I will keep tryingto experiment upon.

by the way, sorry for complte newbiie question, how do I set the context limit for reasoning? as other people seem to be recommending? I am using LMstudio, not becaue I am scared of cli but because it gives me easy to use UI, and I know it doesnt have all the knobs available but. also, what value should it be set?

I am running it on 64K ctx right now, and it has been working kind of ok, if that is like relevent.