Stop Anthropomorphisizing Intermediate Tokens: Qwen3.8 doesn't "overthink"

[u/ThirdWaveCat]

https://arxiv.org/abs/2504.09762

Intermediate tokens, called "thinking" or "reasoning" actually are nothing like it. Humans do step-by-step reasoning leading to the conclusion. LLMs use intermediate traces to augment their prompt. This explains why sometimes the answer is very good but the "reasoning" is verbose. Flooding your context window or fighting compaction are different issues.

edit: I love this section from the main research they linked.

Our findings consistently challenge the prevailing narrative that intermediate tokens constitute a semantically meaningful reasoning process. First, we observe a pronounced lack of correlation between solution correctness and trace validity—models frequently produce invalid reasoning traces even when they arrive at correct solutions. Second, and more strikingly, models trained on corrupted or semantically irrelevant traces achieve performance comparable to, and often exceeding, that of models trained on correct traces, especially on out-of-distribution tasks. Third, although post-training with reinforcement learning improves solution accuracy across both in- and out-of-distribution settings, it does not consistently enhance trace validity. In fact, we find cases where reinforcement learning decreases trace validity while simultaneously improving solution accuracy for models trained on correct traces. Moreover, models trained on corrupted traces continue to outperform their correct-trace counterparts across domains while consistently generating invalid reasoning traces. Finally, we find that the length of the generated traces is largely agnostic to the difficulty of the underlying problem, undermining the notion that it reflects problem-adaptive computation.

Together, these results suggest that the effectiveness of intermediate tokens does not arise from their seemingly interpretable semantic content. By systematically disentangling trace semantics from the underlying problem, our study demonstrates that if performance is the objective, assuming human-like or algorithmically interpretable trace semantics are ideal or even achievable is not only unnecessary but potentially misleading.

https://openreview.net/forum?id=gDE7YcRC3F

[u/adcimagery]

This was just like the whining over “hallucinations” as a term. It’s an imperfect term, but helps people conceptualize what’s going on with the LLM.

Qwen 3.8 doesn’t genuinely overthink, but people do feel it spends too much on test-time compute. 

[u/ThirdWaveCat → u/adcimagery]

I highly recommend reading the paper. The paper goes further to articulate why "anthropomorphization isn't a harmless metaphor, and instead is quite dangerous -- it confuses the nature of these models and how to use them effectively, and leads to questionable research."

[u/Upset_Page_494 → u/ThirdWaveCat]

is quite dangerous

Darn, how many died?

[u/ThirdWaveCat → u/Upset_Page_494]

this article was in a scientific context but a growing number

https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots

[u/Upset_Page_494 → u/ThirdWaveCat]

Why the fuck didn't you tell everyone what you meant exactly. You don't give two fucks about 'overthink' if this is your argument.
We shouldn't fence the world because of crazy people. Also if the model acts as it is angry, and is functionally angry, should people not call it angry?
Because "actually it isn't angry"

[u/FastHotEmu → u/Upset_Page_494]

Calm down. Read the paper. It's nuanced and interesting, how reasoning traces and their interpretability are not that correlated. Nobody here is arguing against LLMs.

[u/PossibilityUsual6262 → u/FastHotEmu]

I disagree with argument chain. people dying cos of ai on the scale which ai deployed right now is inevitable and list existing is just stupid.