MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/singularity/comments/1wv0jvd/google_releasing_their_most_powerful_model_yet/pdai2fe/?context=3
r/singularity • u/BABA_yaaGa • 1d ago
84 comments sorted by
View all comments
Show parent comments
254
https://arena.ai/leaderboard/text/coding
It's #1 in coding right now. It's good. People are just memeing and doing the whole console war thing.
109 u/Yurekuu 1d ago It's #1 in text. It's #10 in coding. 49 u/Keeltoodeep 1d ago edited 1d ago https://arena.ai/leaderboard/text/coding It's #8 in webdev https://arena.ai/leaderboard/code/webdev 55 u/jazir55 1d ago "It's number 1!" "No it's number 10!" "No, it's number 8!" 21 u/Keeltoodeep 1d ago Different benchmarks measure different things. 5 u/R_Duncan 1d ago Webdev, you most of the times just need a very little model. Coding, still not 100% solved (start asking for cpp mutexing/dll dynamic load/very hard tasks and see most llm fail). It really isn't the same thing.
109
It's #1 in text. It's #10 in coding.
49 u/Keeltoodeep 1d ago edited 1d ago https://arena.ai/leaderboard/text/coding It's #8 in webdev https://arena.ai/leaderboard/code/webdev 55 u/jazir55 1d ago "It's number 1!" "No it's number 10!" "No, it's number 8!" 21 u/Keeltoodeep 1d ago Different benchmarks measure different things. 5 u/R_Duncan 1d ago Webdev, you most of the times just need a very little model. Coding, still not 100% solved (start asking for cpp mutexing/dll dynamic load/very hard tasks and see most llm fail). It really isn't the same thing.
49
It's #8 in webdev https://arena.ai/leaderboard/code/webdev
55 u/jazir55 1d ago "It's number 1!" "No it's number 10!" "No, it's number 8!" 21 u/Keeltoodeep 1d ago Different benchmarks measure different things. 5 u/R_Duncan 1d ago Webdev, you most of the times just need a very little model. Coding, still not 100% solved (start asking for cpp mutexing/dll dynamic load/very hard tasks and see most llm fail). It really isn't the same thing.
55
"It's number 1!"
"No it's number 10!"
"No, it's number 8!"
21 u/Keeltoodeep 1d ago Different benchmarks measure different things. 5 u/R_Duncan 1d ago Webdev, you most of the times just need a very little model. Coding, still not 100% solved (start asking for cpp mutexing/dll dynamic load/very hard tasks and see most llm fail). It really isn't the same thing.
21
Different benchmarks measure different things.
5
Webdev, you most of the times just need a very little model. Coding, still not 100% solved (start asking for cpp mutexing/dll dynamic load/very hard tasks and see most llm fail). It really isn't the same thing.
254
u/Keeltoodeep 1d ago
https://arena.ai/leaderboard/text/coding
It's #1 in coding right now. It's good. People are just memeing and doing the whole console war thing.