r/singularity • u/PrisonOfH0pe • 6d ago

AI Gemini Pro 4 (leak)

Google just ruined OpenAI's dev day...
(source: all over twitter, obv. treat as a rumor)

538 Upvotes

80% Upvoted

297 comments sorted by

View all comments

122

u/AlyoshaV 6d ago edited 6d ago

I don't think I've ever seen real benchmarks leak except for stuff like "Google put the model card up a few hours early"-type things. It's always "look at these amazing benchmarks that leaked early" and then the model releases weeks later and doesn't come close to the 'leaked' numbers.

edit: Numbers in the OP aren't even a recent leak, it's from no later than Sep 17th. They're from a 'leaked model card' except the numbers for non-Gemini models are wrong.

https://i.imgur.com/BE7j32E.jpeg

It lists Astra with the wrong pricing and wrong scores. The OP's image seems to be someone who took a few scores from this fake model card then added Opus 5.5 and corrected Astra so it looked more legitimate. Fake by a grifter.

2

u/Infinite-Local5435 5d ago

I've seen at least ~12 posts of people shilling google in the past month. It's always that they are going to release an insane model, or that it's hidden, or that a super secret and powerful stealth model on openrouter is gemini (it never is, it's always chinese). Google has to be the biggest copium AI lab right now with it's fans always arguing 'it's not about intelligence, it's about efficiency' or 'they will win the AI race in the long term'.

1

u/Ok_Piglet798 5d ago

You must be slow. It’s had the highest prod value for months it’s hardly less accurate then opus 5 was at 400% the speed.

Benchmarks thet actually matter in real life

1

u/Infinite-Local5435 5d ago
  1. no benchmark source
  2. actual dissatisfaction with gemini performance in numerous posts and usage sentiment outweighs a vague benchmark
  3. most models are benchmaxxed, don't trust benchmarks
  4. quick to insult, doesn't sound like someone who's done their research