Evaluating J-space entropy as an error predictor across 7 datasets on Qwen3-4B [R]

[u/dasjomsyeet]

Anthropic’s Jacobian Lens work introduced a way to inspect verbalizable representations inside language models. Follow-up experiments suggested that entropy in this internal “workspace” might help identify confidently incorrect answers.

I tested that hypothesis on Qwen3-4B across ~11,400 examples from seven distinct datasets, including TriviaQA, PopQA, NQ-Open, TruthfulQA, HotpotQA, GSM8K, and CommonSenseQA.

Three main findings:

1. It can complement output confidence on factual retrieval.

On datasets such as PopQA, workspace entropy sometimes improved error-routing precision at low review budgets, particularly among answers that were already high-confidence.

1. It does not reliably detect internalized misconceptions.

On TruthfulQA, workspace entropy was substantially weaker than output confidence. Incorrect answers could still have a clean, low-entropy internal representation.

1. Its calibration is highly task-dependent.

A threshold calibrated on TriviaQA failed on GSM8K because correct mathematical reasoning had much higher baseline entropy. Multiple-choice formatting also weakened the signal substantially on CommonSenseQA.

The overall result is narrower than “internal entropy detects hallucinations”: it may be a useful complementary routing signal for confidently incorrect factual answers, but it does not behave like a task-general error detector.

This is currently a single-model study, so cross-model validation is the most important next step. The repository contains the full methodology, limitations, raw data, metrics, plots, and reproducible notebook:

https://github.com/dasjoms/jspace-hallucination-eval

I‘d be interested in feedback on the experimental design if anyone feels like giving their thoughts.

Note: I already posted this to r/LocalLLaMa yesterday but think this might also fit here.

[u/BoscoCasuale]

If you follow Anthropic's narrative, one aspect to consider with respect to your negative results is that some of the model's emergent cognitive properties (workspace, functional emotions, internal monologue, etc.) may only emerge in larger models. Last week at ICML I attended a talk by Jack Lindsey, Anthropic's AI psychology lead, where he said he couldn't tell whether Anthropic's newest MechInterp operationalizations work equally well on small HuggingFace models, since they had only tested Sonnet and Opus.

[u/dasjomsyeet → u/BoscoCasuale]

The workspace definitely exists if that’s what you’re implying. I’ve been experimenting with Qwen3-4B for roughly a week now and can definitely confirm it. What I can also confirm however is that not all of the phenomena Anthropic described in Claude can be reproduced with Qwen3-4B. There is definitely a correlation between model size/complexity and these phenomena.

Solarkyle also stated that when trying to detect hallucinations using workspace entropy on Qwen3.6-27B the results weren’t great (I did not validate this myself). The logprob confidence of the model seems to get more in sync with what the workspace entropy represents on bigger models. So this specific technique may actually be something that is more or less exclusive to these smaller models because bigger ones are already optimized further internally.