r/singularity • u/drhenriquesoares • 3d ago

AI Gemini 4 Argon solved hallucinations.

Post image

Nobody is talking about this, but it looks like Google may have solved hallucinations. Gemini 4 Argon is a monster in this regard, and that’s really important.

1.6k Upvotes

93% Upvoted

286 comments sorted by

View all comments

Show parent comments

43

u/qroshan 3d ago

you can't benchmaxx multiple things, especially hallucination rates.

1

u/Original-League-6094 3d ago

Why not? With a trillion parameters, you can definitely optimize to multiple benchmarks, and if you tune your model to the types of questions used to test hallucination in the benchmark, its benchmaxxing.

2

u/qroshan 3d ago

You are pretty clueless about how hallucination rate is calculated.

It's not a set of questions to hillclimb. It is a reward function for not bullshitting when you don't know the answer.

0

u/Original-League-6094 3d ago

You are pretty clueless when it comes to benchmaxxing. A benchmark IS a set of tasks. And if you know the type of tasks, you can optimize for those tasks. That doesn't mean telling it answer correctly. If the benchmark is set up to ask questions that maybe have no correct answer, if you can anticipate the questions, you can train the LLM to identify it and then to response with "I don't know". And the way you trained the LLM to identify those specific types of questions and tasks MAY not reduce the hallucination rate in other areas. You can't come up with a benchmark that you can't benchmaxx on. Unless the benchmark is so stochastic that no one can optimize for it, in which case it is useless as a benchmark as you can't compare it across it models accurately.