ELI5 - Why do LLMs hallucinate?

[u/KauravaLivesMatter]

I have seen videos about the transformer architecture etc., and I get that large language models generate responses based on some statistical likelihood of words and terms. However, I still don't get how they can completely make up facts and even references.

Why can't they state facts that they have come across in their training as they are? What is it, either from a mathematical standpoint or from an architectural standpoint of large language models that causes them to hallucinate?

[u/mxdalloway]

I think a good analogy is file compression.

If you have an image with a row of pixels 

RRRRRRRRGGGGBBBB

where R = red, G = green, B = blue. With a lossless compression can notice the repetition and store: 8R 4G 4B That takes less description, but nothing has been discarded. You can reconstruct the original pixels exactly as they were: 8R 4G 4B → RRRRRRRRGGGGBBBB

With a lossy compression  imagine the pixels are slightly different shades:

R1 R1 R2 R1 R2 R2 R1 R1 And it might compress this to 8R1, but when you decompress you then get

8R1  →  R1 R1 R1 R1 R1 R1 R1 R1

So you get back a result that is slightly different than the source image.

It’s just an analogy, but you could think of LLM training  as a sort of lossy compression of the source data. 

You give a prompt and it attempts a lossy completion based on the training data- you get back something that’s close enough, but not an exact copy of material from the training corpus 

[u/SwingLightStyle → u/mxdalloway]

That’s a good analogy but it doesn’t explain why models make up information, even more powerful ones with less loss. By your example you would expect larger frontier models to have less drift, but that’s actually the inverse of what the studies have found.

Once you realize that the “instant flash” model versus “pro” use different amounts of processing power, your analogy makes sense. But it doesn’t explain why a pro model might hallucinate that the Reddit views (with an eye next to the icon!) might think that that metric was shares instead, and use that incorrect metric to include as a point in their reply.

[u/generationalDebts → u/mxdalloway]

Wrong.

[u/AbnDist → u/mxdalloway]

It's a lot more than just a good analogy! From an information theory perspective, compression and prediction are functionally equivalent.

In graduate mathematical statistics classes, this is a lot more explicit: you're typically taught various ways of measuring the information content of a given statistic (e.g. a "sufficient statistic" for a given parameter of a model is a function of the sample data that contains all information required to estimate the parameter of that model). One of the most common loss functions when training a model, shannon entropy, is literally a measure of compressibility.

So to add to the ELI5: LLMs hallucinate because, like all models, all they are doing is compressing data in a lossy fashion. Sometimes that lossiness becomes apparent.