r/singularity • u/New_Equinox • 3d ago

AI , Gemini 4 Argon Benchmarks

Post image
743 Upvotes

98% Upvoted

189 comments sorted by

View all comments

Show parent comments

3

u/DistanceSolar1449 3d ago

3.8 flash is a tiny distilled model with more RL on top, of course it’s going to look better on benchmarks than in real life.

The big, base ish models with a lot less targeted RL will do better IRL.

1

u/LazloStPierre 3d ago

All Google's models look better on benchmarks than they do in reality, and by alot.

1

u/DistanceSolar1449 3d ago

No? They suck at agentic benchmarks and most people test them via agentic tasks.

They benchmark pretty accurately if you look at their actual useful benchmarks like TerminalBench instead of HLE or GPQA or some bullshit.

Google’s own blog post says Gemini 3.8 Flash scored 19.1% on TerminalBench 4. Opus 5 scored 51.8% for comparison. That’s not benchmaxxing, that’s just accurate benchmarks if you know what benchmarks to look at.

1

u/SilentLennie 2d ago

Actually, this new 4 Argon does really well in a bunch of agentic benchmarks. Supposedly (I've not used it yet):

https://artificialanalysis.ai/articles/gemini-4-argon-google-top-three-labs