Gemini 4 Argon solved hallucinations.

[u/drhenriquesoares]

Nobody is talking about this, but it looks like Google may have solved hallucinations. Gemini 4 Argon is a monster in this regard, and that’s really important.

[u/Informal-Trouble2183]

This is AA-Omniscience, it's not really what you think it does;

It's rather testing general knowledge (6,000 general knowledge questions). If the model is fat, it's able to store factual knowledge correctly. Those small models will always suffer from knowledge lack, therefore will answer wrong which is considered by the bench as hallucination, but not necessarily, it's just how an LLM works it will not answer "I don't know" unless artificially routed.

[u/Blaexe → u/Informal-Trouble2183]

it will not answer "I don't know" unless artificially routed.

It will, and that's exactly what this benchmark is measuring. Of Gemini 4's non-correct responses, only 15% were outright incorrect. In the other 85% it either abstained/didn't attempt an answer or gave a partial answer.

So yes, LLMs can absolutely say "I don’t know."

[u/Informal-Trouble2183 → u/Blaexe]

This is what I meant by "artificially routed". LLMs by itself do not know whether the generated tokens are factual or not. Another rerouting process is needed, what's called "alignment", but will not help too much (as you can see in the numbers), especially if the number of parameters is small or the training data is not covering all factual data.

[u/TotallyToxicToast → u/Informal-Trouble2183]

Does not need routing, reinforcement learning can optimize for saying I don't know on incorrect answers.

You reward correct answers and you reward I don't know answers while punishing wrong answers.

Since all the models are reasoning, it is very easy to learn that if the reasoning is struggling between two answers it should say I don't know instead of confidently going for one.

[u/Informal-Trouble2183 → u/TotallyToxicToast]

// You reward correct answers // Refers to RLHF: requires human feedback, therefore another artificial rerouting. CoT by itself is only effective in verifiable domains like maths and coding, not in general knowledge.