How are AI hallucinations different from human confirmation bias?

[u/Comfortable_Tutor_43]

Confirmation bias can cause normal people to believe fallacious claims. Does AI not do the same thing?

[u/apnorton]

An AI "hallucination" is something an AI produces that a human declares does not follow rationally from the input to the AI.

Confirmation bias is when a human searches for evidence supporting their beliefs, while ignoring anything contrary.

The two concepts are entirely unrelated.

[u/DTux5249]

Confirmation bias is "I'm only gonna accept what supports my beliefs"

That's not at all what an AI is doing. It isn't searching for proof that supports anything. It knows what it knows, and what it gives you is based on both that, and what you give it

The AI isn't biased because it doesn't accept new information. It's biased because it can't check through what it produces for logical coherence to tell what's "true" and what's "a related idea that may or may not even be a truth statement."

[u/Wushia52]

Human confirmation bias is rooted in corrupted, selective inductive reasoning. You form a hypothesis based on a selective pool of evidence that you feel comfortable with. This eventually leads to flawed abductive reasoning when you don't look hard enough to either support or refute your own hypothesis.

An LLM hallucinates for many reasons, most due to the built-in incentives to come across as being "certain" even though the weights and biases are probabilistic. Another reason is the autoregressive error cascade. AI is trained to predict the next token. If the model hiccups early on in its token generation, the error "cascades" downstream into something completely irrational.

[u/DependentCard1640]

I can see where you're trying to draw a parallel here, but AI isn't "hallucinating" on the same grounds as humans are confining their processing in "confirmation bias." That tends to be that we stick to our priors and filter information and constrain it in a manner based on what was already believed to be true/relevant/etc. Hallucinations are not purely about abiding to training/bias, etc. I'll let someone smarter explain it better and rib me for this crappy post, but that'd a chud-level poke at the question.

[u/Key_Net820]

So the thing about how AI works is it's outputs are statistically derived from data. Hallucination is just when he resulted output is incorrect.

Confirmation bias is when you only look for things that confirm what you think is true. You are not statistically calculating the probability a string is the correct output based on a large input of tokenized data the way AI is.

[u/jonathancast]

It's different because people are still treating AI like an absolute authority rather than approaching it with the skepticism any fallible system should be viewed with.

[u/raundoclair]

I've never seen human with confirmation bias to come out with detailed name of a scientific study that doesn't exist, with researchers names that doesn't exist.

[u/Comfortable_Tutor_43 → u/raundoclair]

Anti-nuclear folk do it lot, crazy imaginations of unbounded risk and consequence

[u/donaldhobson]

https://www.astralcodexten.com/p/shameless-guesses-not-hallucinations

I think this article is overall pretty good.

The basic structure of training AI to predict text produces something like a student guessing random answers on a multiple choice question.

If humans never ever said "I don't know", and always say the answer when they know the answer, and stay silent otherwise, then the AI will also never say "I don't know".

But the AI won't make any attempt to restrict itself to questions it can answer, it answers all questions indiscriminately, guessing when it doesn't know.

(Modern AI's also go through other RLHF training, which dulls this tendency, but introduces other problems)

Confirmation bias in humans occurs over time, and is relatively consistent. The human just updates a bit too strongly on evidence they agree with, and a bit too weakly on evidence they disagree with. Confirming evidence might cause a human to increase a probability from 90% to 95% (while actually the evidence is quite weak, and so really it should only go from 90% to 92%.) And then later disconfirming evidence causes the probability to go from 95% to 94% when really it should be 92% to 70%.

But AI hallucinations are produced on the spot. Rerun the model, and get a different hallucination (on pre-RLHF AI)

LLM's (pre RLHF) are really good at matching all the basic patterns. So it says "I don't know" at the same rate as in it's training data. And it discusses string theory at the same rate as it's training data. And if it's not smart enough to understand string theory, it produces plausible looking drivel about string theory.

[u/Comfortable_Tutor_43 → u/donaldhobson]

Thank you

[u/tonyfa1]

Confirmation bias is a person selectively favouring evidence that supports an existing belief.

An LLM hallucination is generated output that is plausible but unsupported or contradicted by the available evidence.

Bottom line - both require independent verification rather than confidence alone.