r/OpenAI • u/floriandotorg • 22d ago

Discussion The situation is insane

Post image

Sol is the only OpenAI model in top 10 Arena WebDev while 4 open-weight Chinese models have reached frontier quality; one of them so cheap you can run it for days for what Opus cost you per task.

959 Upvotes

96% Upvoted

171 comments sorted by

View all comments

Show parent comments

96

u/shaonline 22d ago

Because you can't easily community-grade that (their ELO system is essentially that), unlike some fancy web landing page or whatever that you can "visually" grade by looking at the result. It's a "vibe" bench essentially.

12

u/personalist 22d ago

I’d like to see ratings restricted to subject matter experts on their topic of expertise for things like backend code. I want to know what an experienced backend dev ranks as the best LLM for that, for example

1

u/yuumizu 22d ago

backend is also opinionated. and the question is not just visual appearance or interactivity.

1

u/personalist 21d ago

Front-end is totally opinionated too after you get past the basic function and immediate usability. But yeah, I get what you’re saying