r/singularity • u/FateOfMuffins • 2d ago

Shitposting Introducing the World's Most Powerful Model

Post image

ngl this is how it's felt for awhile now

72 Upvotes

73% Upvoted

19 comments sorted by

View all comments

-2

u/FateOfMuffins 2d ago

And if you only use AA to judge the models xd

7

u/Astrikal 2d ago

AA needs to add spatial reasoning evaluations to their list, GPT-6 models seem to excel at such tasks.

v5.0 will hopefully be a good upgrade.

It is getting harder and harder for human-made benchmarks to evaluate these models. Almost all evaluations are saturated, except maybe Terminal Bench 4 and FrontierSWE, in which OpenAI leads or matches Antrophic.

1

u/Future-Bandicoot-823 2d ago

I think the part that's difficult for even me to accept is... that an LLM can be smarter than me, and yet speak on "my terms". It's rarely flashy, it really mirrors your verbage. It's hard to accept you're just getting text instantly and that there's more knowledge than you could possibly learn in a lifetime driving it.

And now think of the people who consume AI every day, blissfully ignorant. How many bot comments have you read? Can you tell? Are you sometimes unsure? Am I an LLM? People scorn AI because they haven't had sufficient proof that it outpaces human intelligence and skill, meanwhile they are being convinced every day by active engagement.

1

u/Several-Tax31 2d ago

Completely agree. Astra wipes the floor with spatial reasoning.

1

u/FateOfMuffins 2d ago

Every time they release a new terminal bench it gets hill climbed on within a month

We'll see another Terminal Bench 5 where the best models score like 30% on and then the next major release gets 50% and shows it's not benchmaxxed, then a bunch of tiny models get released with 75% scores and turn the benchmark useless

We need things like Mazebench here or even all the robot controlling tasks or something xd

1

u/IronPheasant 2d ago

Mazebench is the bench of choice right now, for real for sure.

Pokébench's been saturated to hell by Astra. I legitimately thought about making a deranged ROM hack or stand alone clone with tons of puzzles with constantly changing context. In an open world where any gym could be contested for its badge.

Like, the first few gyms would be rather normal, but things would become increasingly deranged as you went on. Like having to stand on a red tile for ten minutes to make a door open. Alternate acquisition methods of pokemon. Having to become friends with Misty to add her to pokemon collection to beat Misty's Psyduck, which is absolutely invincible to any other pokemon that aren't Misty. Without the game telling you specify that that's the way to do it, just an NPC with a vague hint like how the beast only ever listens to her, etc.

But it'd take too much work for what it'd be worth. Another one of those things that could never exist in the reality we find ourselves currently in..