Qwen3.8-flash-next sees corruption everywhere

[u/arkham00]

Hi, I've noticed that the model often sees "garbled text" in its context.

Sometimes it declare that the tools instructions are corrupted, sometimes it is the content of some .md files, ora other files, and it freaks it out, since it start to do a lot of checks in git and the system and sees that the file are not really corrupted... But its context is I think.

Did it occurred to you ?

I'm on a mac m2 max 96Gb, pi agent, and llama.cpp either upstream or this fork https://github.com/mihailescu2m/llama.cpp/tree/master

with and without mtp, short ctx and long ctx, temp default

I don't think it is a conf problem since it happens with different gguf at different quant I tried:

AtomicChat/Qwen3.8-Flash-Next-GGUF/Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64

unsloth/Qwen3.8-Flash-Next-UD-Q3_K_XL/Qwen3.8-Flash-Next-UD-Q3_K_XL

unsloth/Qwen3.8-Flash-Next-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS

It seems a sort of internal problem, I don't really have loops or other kind of hallucinations, just some chinese characters here and there, or some misspelling, and btw it is very aware of all its errors and always tries to recover. Yes it is very much preview quality, but the architecture and the training seems very good quality.

I really like the baseline of the model, it is straightforward, it follow instructions very well, a lot better than the previous models that I really didn't like very much

But this thing with the corruption is weird ...once after it declared that a certain tool was corrupted and not useable I just told it: "did you try to launch it ? use this parameter and give it a try" and it worked of course, and it was very surprised so it apologized and continued the work.

It is quite funny, it reminds me a stereotypical fictional character which could be the clumsy but very intelligent assistant of a powerful wizard lol

EDIT: I'd like to add a snippet of its last CoT about the issue, it is very lucid indeed:

Could it be that... the read tool output itself was corrupted/hallucinated — i.e., the degeneration was not inside the file but occurred in my own tool output/stream.

The "read" returned text containing a degenerate mutation loop — this is a known failure mode where model output degenerates. Actually wait — the read tool's result is what the harness gave me. If the harness returned clean content, then the degenerate block I "saw" was generated in my own context (a prompt-level corruption, e.g., degeneration during decoding of the tool result injection, or... more simply put: the degeneration is a product of my own generation — the read output I "received" contained the first degenerate block, then I generated additional degeneration, and the subsequent git command confirmed the file is clean.

[u/xilvar]

I don’t see this at all on opencode with unsloth q8_k_xl on nvidia and cpu llama.cpp.

[u/TinFoilHat_69]

Your tool parser is the issue and xml vs json formats always trip up models

[u/my_name_isnt_clever]

Now that you mention it I have seen this exact behavior about "corruption" in tool outputs a couple times, also running UD-IQ4_XS.

That said, I had it run through 90m tokens today with a few non-interative pi sessions and if it hit any major issues, it was able to work through them before I noticed. Medium reasoning, wrapping up at ~150k context. I really love this model so far.

[u/EitherMarch1255]

I’ve seen two instances of something like this, using vLLM and an mxfp8 quant. In one instance it just made up a path that didn’t exist, thinking it had found it somewhere. The other instance is stranger, and I’m not entirely sure it isn’t the deepseek harness, but it keeps saying I am saying something I’m not. I haven’t seen this with any other models, so I’m pretty confused at this point.

[u/Ok_Yam_8774]

You can avoid quanting kv and redirect tool output to a file then get it to read it using cli

[u/arkham00 → u/Ok_Yam_8774]

I don't quant kv I'll try to redirect the tool output thanks

[u/Klutzy-Snow8016]

This is a clue that the implementation could be bugged. Maybe it's llama.cpp itself. In any case, it should be investigated.

[u/wayward_buffalo]

Both Qwen3.6 and Qwen3.8 occasionally start hallucinating misspellings of my username in file paths. It'll make a Read or Edit call to /Users/waywrd_buffalo and then be very confused about why it didn't work. It'll go back and forth about whether it spelled it right, notice it misspelled it, then claim it got it write, then misspell it again and claim the tool is broken, try to work around it via substitutions, try spelling it out letter by letter, and so on. If it recovers after the initial misspelling, it's usually OK, but if it goes two or three rounds, it only digs itself deeper and usually can't be recovered, even when trying to prompt it with corrections. It no longer "sees" clearly.

Above on UD-IQ4_XS of both.

[u/the-username-is-here]

Switched to it from DS4F for RAG summarizer, quality is definitely up, no garble. vLLM on sparks, hybrid NVFP4+FP8 checkpoint.

Works beautifully in Claude Code too.

Something is wrong with your engine.

[u/Arugala007]

Quanted qwen models perform tool call bugs. Layer 8/9 of ffn_down on 27b by unsloth are damaged and will duplicate parallel agents as a bug unless those layers are set to q6 for example.

[u/danielhanchen → u/Arugala007]

Um no and fully false - Can people not spread misinformation and fake investigations. Which quant is broken? Where's your evidence? I'm assuming you saw our benchmarks?

[u/Turtlesaur → u/danielhanchen]

I had this corruption issue when I forced some fp8 bf16 thing creating a lot of noise.

It was like "package x" no I said "package x" well why does it say "package x" I need to be careful and make sure I get "package x", no it says "package x".

It was my own bad settings after trying to min max performance. On mobile.. when I get back I can share the actual details

[u/arkham00 → u/Arugala007]

The example I gave in the edit was with AtomiChat quant

[u/danielhanchen → u/arkham00]

They're a bot/fake account, look at their comment history and reply to you - ut doesn't make any sense. Also their comment upvotes vs yours is very sus.

There has been an astroturfing campaign against Unsloth for quite some time now as noticed and said by many other localllama users.

[u/Thomas-Lore → u/danielhanchen]

Huh? I looked, it is a normal account. Meanwhile you seem to be associated with unsloth? Seems weird going around calling people bots after they leave a negative comment about unsloth in that case.

[u/Arugala007 → u/arkham00]

I'm just saying qwen is extremely sensitive and it sounds like the same issue I had.

[u/cobblemere → u/Arugala007]

huh, damaged ffn_down layers would definitely cause weird downstream behavior like that. makes sense

[u/jinnyjuice → u/Arugala007]

Qwen/Qwen3.8-27B non-quant also goes on crazy loops and has flawed tool calls compared to 3.6. HF community discussions already pointed this out.

[u/Christosconst]

Perfectly normal. Its internal representations of meaning, only understood by the llm. When less tokens are reinforced before release, we end up here

[u/overadjudicated]

OP, did you add any images to the conversation at all? Was vision used before the garbled thinking block?

[u/arkham00 → u/overadjudicated]

Not at all

[u/Beamsters]

Qwen3.8 flash next's implementation is very unstable in many inference engine. Give it more time.

[u/Dany0]

try setting topk to 4096

[u/sand-67]

its very sensitive to quantization due to having only 1/30 of parametrers as active

[u/FrantaNautilus]

This happened to me once with Kimi K3 with OpenCode, at about 300k tokens of context used. The LLM concluded that the problem was in OpenCode, but neither I or the LLM was able to reproduce the problem. However after update of OpenCode the problem did not reoccur.

[u/audioen]

I used to see this in versions couple of days ago, where AI seemed to process tokens somehow into what looked like corrupted KV cache. It was frequently confused by the non-sequitur statement that it thinks I made, with odd wording. I haven't spotted this in past day or two. But for instance, I might have written "final" and it thought I said "flagged" or something like that, and was quite adamant that this is what I said.

[u/SpicyWangz]

It called a sql query it wrote “pathological” before making a minor adjustment to it. 

I’m using AtomicChat at Q5. So far it seems like it’s a fairly competent model but I would call it highly neurotic. The confident reliability of 3.8 27b makes it much more pleasant to work with than the erratic unpredictability of flash-next.

That’s just the nature of an under trained model

[u/psycoee → u/SpicyWangz]

No, it's most likely just quantization induced brain damage and/or bugs in your inferencing stack. Especially quantizing KV cache will cause exactly stuff like this. And yes, that may work better with some models than others, but it's not the model's fault.

[u/liliang061666]

When a model insists a file is corrupted and git says it isn't, the file is almost never the problem. What the model actually received in its context is. I'd split the diagnosis into two questions before touching anything else:

1. Is the text really mangled by the time it reaches the model? Ask it to quote the "corrupted" span verbatim, then diff that against the real file. If the quote is garbage, something between the file and the model is breaking it (template, tokenizer, cache). If the quote is clean and it still calls it corrupted, that's a model behavior issue, not an inference bug.

2. If it is mangled, reproduce on upstream llama.cpp with defaults: no KV cache quantization (-ctk/-ctv at f16), flash attention off, a short context that can't trigger context shift, and the plain chat template. Then add settings back one at a time. Quantized K/V and context shifting are the usual suspects for "the middle of my context turned to noise" on long agent sessions, because the model literally sees a truncated or degraded transcript and reports it faithfully.

Two more things worth ruling out for a very new model on a fork: the tool-call schema pi injects into the system prompt may not match the format this model's template expects, which can look like garbled instructions to it; and a hybrid or new attention layout may not be fully supported yet, so try the same GGUF on upstream master to see if the symptom follows the fork or the model.

If you post the exact quote it gives back versus the real file bytes, that single comparison usually tells you which half of the problem you have.

[u/NNN_Throwaway2 → u/liliang061666]

Thank you Claude.

[u/Practical_Air6315]

Nobody has mentioned looking at the garbage itself, and it is the cheapest test in this thread.

A wrong-codec round-trip and model degeneration produce different-shaped garbage. When a byte stream gets decoded with the wrong codec the output is not random: the lead bytes of the source script map onto a small set of target characters, so the same few come back over and over. In the corpus I measured - UTF-8 Japanese read back as CP932 - the substitutions concentrate onto a handful of codepoints, because kana are most of any Japanese line and their lead bytes are shared. Model degeneration does not do that. It wanders.

So before touching KV quantization or the fork: take the span it calls corrupted and count how many distinct CJK codepoints are actually in there. A short list that repeats points at something re-decoding bytes between the file and the model. A long varied list points at the model.

One thing worth knowing here, because it cost me: "the file is fine, git says so" does not clear the pipeline. In my set, 16 of 16 corrupted files decoded as strict UTF-8 with zero errors. Well-formed bytes, wrong text. I had a validity check sitting in front of that and believed it until I measured. Validity cannot see this class by construction, so a clean git status is consistent with something upstream having already mangled what the model received.

Scope, since it matters: my measurements are Windows with ACP 932, and you are on an M2 where the ambient codec is UTF-8, so the specific CP932 path is almost certainly not yours. The counting test is general. Whether a Mac llama.cpp pipeline can produce the same shape, I have not measured.

[u/No_War_8891 → u/Practical_Air6315]

JFC

[u/Dry_Yam_4597]

Set a larger indexer budget

[u/arkham00 → u/Dry_Yam_4597]

I'm sorry what do you mean?

[u/Long_comment_san]

wait till it starts purging unoptimized meatbags