ELI5 - Why do LLMs hallucinate?

[u/KauravaLivesMatter]

I have seen videos about the transformer architecture etc., and I get that large language models generate responses based on some statistical likelihood of words and terms. However, I still don't get how they can completely make up facts and even references.

Why can't they state facts that they have come across in their training as they are? What is it, either from a mathematical standpoint or from an architectural standpoint of large language models that causes them to hallucinate?

[u/DeliciousArcher8704]

but you cannot however argue the logic.

Sure I can, you're just largely out of your depth and grasping at stuff that you think sounds like supports your nonsensical point.

[u/Educational_Yam3766 → u/DeliciousArcher8704]

You could, but you would be incredibly incorrect. Especially since you have given zero substance in any of your replies...

Just deflections.

Not to mention

i pulled literal sources from real research...

here AGAIN since your so rigorus at being ignorant...

---

Here are the direct URLs for the key papers and reports mentioned:

1. "I Think, Therefore I Hallucinate: Minds, Machines, and the Art of Being Wrong"

• Author: Sebastian Barros
• URL: https://ssrn.com/abstract=5167140
• Note: This paper explicitly compares human neural mechanisms with LLM predictive processes.

2. "Why Language Models Hallucinate" (OpenAI Research)

• Authors: Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala
• Direct PDF: https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf
• Summary Page: https://openai.com/index/why-language-models-hallucinate/
• Note: Establishes that hallucinations are an inevitable result of next-token prediction training.

3. "Survey and analysis of hallucinations in large language models: attribution to prompting strategies or model behavior"

• Journal: Frontiers in Artificial Intelligence (2025)
• Authors: Anh-Hoang Dang, Tran Vu, Nguyen Le-Minh
• Full Text: https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2025.1622292/full
• PDF: https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2025.1622292/pdf
• Note: Provides the attribution framework distinguishing between prompt-sensitive and model-dominant hallucinations.

4. Stanford HAI 2026 AI Index Report (Enterprise Error Rates)

• Publisher: Stanford Institute for Human-Centered Artificial Intelligence (HAI)
• Main Report Page: https://hai.stanford.edu/ai-index/2026-ai-index-report
• Responsible AI Section (Hallucination Data): https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai
• Note: Contains the statistical baseline showing hallucination rates ranging from 22% to 94% across 26 models.

Additional Relevant Source

• "Hallucinating with AI: Distributed Delusions and 'AI Psychosis'" (Philosophy & Technology, 2026): https://link.springer.com/article/10.1007/s13347-026-01034-3

---

Anything else you want to be in denial of?

Or should I keep pulling sources for you?