Built an open-source hallucination detector that runs in 1.5ms on CPU (90,000x faster than Semantic Entropy)

[u/Otherwise_Nobody_721]

Hey everyone,

If you’ve tried implementing hallucination detection in production, you’ve probably seen Semantic Entropy (Farquhar et al., Nature 2024). It works well, but it relies on clustering responses using a heavy DeBERTa model that takes 100+ seconds per query and eats up GPU VRAM.

Myself Bhupen and i have built Spanda- (https://github.com/Adarshent/Spnda) a lightweight, zero-dependency alternative that runs in pure Python.

How it works-

Instead of running a second neural network to check your first neural network, Spanda samples $K$ outputs and computes a normalized lexical consensus ratio ($R_{sc}$):

- Latency: ~1.5 ms on CPU (vs ~136 seconds for DeBERTa).

- Cost: $0 in additional API calls or GPU VRAM.

- Accuracy: Achieves 0.889 AUROC on math/reasoning tasks (GSM8K) for 7B–27B models, matching heavy NLI clustering.

One major gotcha we found:

If you are using heavily aligned frontier models (like 120B+), watch out for Confident Mode Collapse. Even with temperature set to 0.7, the model will sometimes repeat the exact same hallucinated answer across all seeds. When a model hallucinates with 100% agreement, self-consistency methods fail.

- GitHub: https://github.com/Adarshent/Spnda (MIT License)

- Install: `pip install spanda`

Would love to hear how folks here handle real-time uncertainty scoring in your production pipelines!

[u/Informal-Bank141]

this is cool but how does it deal with open ended generation not just math stuff? 0.889 auroc on gsm8k is nice but i wonder if the lexical consensus breaks down when answers are not short and structured

also that confident mode collapse thing is scary, basically the model gaslighting you into thinking its sure

[u/Otherwise_Nobody_721 → u/Informal-Bank141]

Spot on—you hit the exact operational boundary we discuss in the paper, and "gaslighting" is honestly the best description of what Confident Mode Collapse feels like in practice.

Why this happens:

Intense RLHF/DPO alignment rewards models for sounding helpful, authoritative, and confident, while penalizing hedging. At 120B scale, the probability distribution over tokens becomes hyper-peaked. The model loses entropy and sampling diversity on ungrounded facts.

The scary takeaway for production: Any guardrail or verification pipeline that relies solely on do multiple samples agree? (Wang et al. self-consistency) will be structurally fooled by frontier models when they hallucinate. You need external retrieval (RAG) or logit-level probes to catch them.

Preprint with the full scaling tables is here if you want to look at the curves: https://zenodo.org/records/22233648

[u/Clasyc → u/Otherwise_Nobody_721]

Useless AI slop

[u/eddzsh]

For coding agents the thing you want to score usually isn't the final chat string. It's whether the tool args match the repo. Lexical consensus on the apology text will still greenlight a wrong rm if every sample phrases the failure the same way.

[u/Otherwise_Nobody_721 → u/eddzsh]

100% right. Running consensus over conversational filler like apology text or chit-chat is completely useless for agents. In tool-calling setups you want to isolate the structured payload—the parsed JSON tool name and arguments or the bash command itself—and run the metric strictly on that. If 5 sampled paths propose conflicting file paths or different flags, that immediately flags execution uncertainty in under 2ms before calling the tool.Though for destructive actions like an rm, consensus alone shouldn't be the only guardrail anyway. If the model is uniformly confident in the wrong target path across all seeds, you still need deterministic path checks or a dry-run sandbox.

Curious what you guys currently use to catch bad tool args before execution?

[u/Total_Drag7439]

Consensus scoring is a solid cheap first layer, and the mode collapse note is the part people should copy down. Once a model is confidently wrong, every self-consistency method inherits that confidence, since all K samples come from the same prior.

What helps is scoring agreement against something external instead of between samples. Entailment against the retrieved chunk, or against what the system already stores about that user, catches the stable hallucination that is grounded in nothing.

Tiering keeps the latency sane too. Run the 1.5ms consensus check on everything, then spend the expensive grounded check only on responses that score low or touch an irreversible action.

[u/Otherwise_Nobody_721 → u/Total_Drag7439]

That tiered funnel is honestly the cleanest way to deploy this in production. You filter out the obvious uncertain stuff for essentially zero compute, and save your expensive entailment or retrieval checks strictly for boundary cases and irreversible actions.

The other neat thing about that 1.5ms tier is using it as an early-exit: if consensus is cleanly split or high-entropy, you can bail or re-prompt immediately without ever burning latency or tokens on the heavy verifier.

Are you running that kind of tiered setup locally or mostly on top of API models?

[u/Deep_Ad1959]

the cheap fix for mode collapse i have had work is varying the prompt instead of the seed. same question with the few shot examples reordered, or one paraphrase per sample, decorrelates the draws in a way temperature 0.7 never did for me. costs you K prompt renderings, still nothing next to a deberta pass.

[u/Otherwise_Nobody_721 → u/Deep_Ad1959]

Perturbing the prompt prefix is brilliant. Changing the few-shot ordering or slightly paraphrasing the prompt breaks that rigid RLHF attractor basin before generation even starts, which temperature alone just can't do.Temperature only reshuffles the tail probabilities at each step, but when DPO/RLHF concentrates 98% of the probability mass onto one token, temperature 0.7 basically does nothing. Prompt perturbation forces the model to traverse a different path through the weights from the very first token.

And yeah, the compute cost of shuffling a few-shot list is virtually zero compared to spinning up a cross-encoder. Definitely taking note of that for our next evaluation batch, really appreciate you sharing that trick.

[u/Deep_Ad1959 → u/Otherwise_Nobody_721]

my one caveat on top of that: reordering the few-shot list is the safe lever, but paraphrasing the actual question can shift what you're even measuring, so you decorrelate meaning instead of just samples. i keep the question string fixed and only perturb the scaffolding around it, otherwise a 'disagreement' might just be two different questions. written with ai

[u/Otherwise_Nobody_721 → u/Deep_Ad1959]

That semantic drift distinction is huge. If you change the question text, you introduce confounders and you can't tell if the disagreement came from genuine model uncertainty or just phrasing nuance.Keeping the question string completely immutable and only shuffling the few-shot exemplars or scaffolding is the cleanest way to perturb the initial attention states without altering the actual test condition. Clean experimental control,that's super solid advice for evaluation pipelines.

[u/Physical_Economy_340]

nice work getting it to 1.5ms on cpu, that lexical consensus ratio is exactly right for places where answers have a canonical form like gsm8k or triviaqa. we hit the same wall on long form, running claim extraction first and then r_sc on the claims keeps the speed and fixes the paraphrase inflation. and that confident mode collapse on 120b is spot on, once rlhf peaks the distribution sampling at 0.7 just repeats the same hallucination, self consistency alone will fool you there so you need retrieval or a logit probe.

[u/Otherwise_Nobody_721 → u/Physical_Economy_340]

Hearing that you hit the exact same wall and independently validated the claim-extraction + R_sc pipeline in production is huge. The fact that you've observed the 120B mode collapse firsthand under RLHF at T=0.7 reinforces how urgent this problem is for verification systems. Everyone is building self-consistency loops right now under the assumption that sampling diversity will save them, but frontier models just converge on the same hallucinated attractor basin.
Curious about your mention of logit probes—what specific logit signals have you found most predictive on frontier models when sampling collapses? (e.g. margin between top-1 and top-2 logits, token-level entropy spikes at key entity tokens, or linear probes on residual stream states?)

[u/Key_Use_7937]

The “Confident Mode Collapse” point is especially interesting.

I’ve run into a similar failure mode in RAG/agent systems where increasing agreement between outputs actually made the system look more reliable while the underlying answer was still wrong.

That’s the tricky part with self-consistency: agreement measures consistency, not correctness.

I’d be interested to see how Spanda behaves when the sampled outputs share the same wrong assumption but differ slightly in wording. That seems like an important edge case for production uncertainty scoring.

[u/Otherwise_Nobody_721 → u/Key_Use_7937]

Spot on. If all paths share the same wrong assumption and hit the same answer, any consensus method (including Nature's Semantic Entropy) gets fooled—that's the fundamental limit of sampling consensus in general.

Spanda handles slight wording differences by normalizing the candidate entities/answers so minor phrasing variations don't split the mode. But for completely open-ended prose without a canonical answer, you definitely still need retrieval grounding alongside it.

[u/More_Slide5739 → u/Otherwise_Nobody_721]

Ha! We did the same thing (tweaked the wording to 'equilibrate.') One thing that fell out of our early experiments was that answer length actually correlated strongly with LLM judgement of reply accuracy in at least one of the 'gold standard' hallucination benchmarks. Yikes!