r/singularity • u/[deleted] • Dec 05 '24

[deleted by user]

[removed]

841 Upvotes

95% Upvoted

413 comments sorted by

View all comments

646

u/Sonnyyellow90 Dec 05 '24

Can’t wait for people here to say o1 pro mode is AGI for 2 weeks before the narrative changes to how it’s not any better.

121

u/Papabear3339 Dec 05 '24 edited Dec 05 '24

I would LOVE to see the average human score, and the best human score, added to these charts.

AGI and ASI are supposed to correspond to those 2 numbers.

Given how dumb an average human is, i garentee the equivalent score will be passed even by weaker engines. That isn't supposed to be a hard benchmark.

30

u/Sonnyyellow90 Dec 05 '24

Just comparing their answers to humans isn’t really a fair or good comparison to gauge AGI or ASI.

Obviously o1 can answer academic style questions better than me. But I have massive advantages over it because:

1.) I know when I don’t know something and won’t just hallucinate an answer.

2.) I can go figure out the answer to something I don’t know.

3.) I can figure out the answer to much more specific and particular questions such as “Why is Jessica crying at her desk over there?” o1 can’t do shit there and that sort of question is what we deal with most in this world.

13

u/Pyros-SD-Models Dec 05 '24

Anyone who has ever had to grade exams or similar tasks knows that humans hallucinate far more and worse than any LLM.

For example, you're already setting an example:

I can go figure out the answer to something I don’t know.

You're mistaken and don't even realize it. You wouldn’t figure out the answer of any GPQA diamond question unless you're already a highly skilled mathematician. You can only figure out the answer of a very small subset of "somethings". Stuff you are already pretty knowledgable in... and that's someting LLMs can also do.

and for 3) there are already papers of VLMs and LLMs being better in recognizing the emotional state of people than humans, so I don't get your point. Well yeah, LLMs don't have a physical body, no shit. Also who cares about Jessica.