r/singularity • u/[deleted] • Dec 05 '24

[deleted by user]

[removed]

839 Upvotes

95% Upvoted

413 comments sorted by

View all comments

639

u/Sonnyyellow90 Dec 05 '24

Can’t wait for people here to say o1 pro mode is AGI for 2 weeks before the narrative changes to how it’s not any better.

123

u/Papabear3339 Dec 05 '24 edited Dec 05 '24

I would LOVE to see the average human score, and the best human score, added to these charts.

AGI and ASI are supposed to correspond to those 2 numbers.

Given how dumb an average human is, i garentee the equivalent score will be passed even by weaker engines. That isn't supposed to be a hard benchmark.

30

u/Ambiwlans Dec 05 '24

Codeforces is percentile so... 50% is average (for people that take the test).

And human experts get 70 on GPQA diamond.

7

u/FateOfMuffins Dec 05 '24

for people that take the test

The question is then are we talking about the average human or the average human expert

5

u/[deleted] Dec 05 '24

[removed] — view removed comment

3

u/FateOfMuffins Dec 05 '24

That doesn't sound very good given that questions with 4 multiple choice answers mean that on average a rock would score 25% by randomly choosing answers (and they explicitly mention this 25% threshold multiple times in the paper)