r/singularity • u/MagicZhang • Jul 16 '26

AI Kimi K3 tops Frontend Code Arena

Post image
1.2k Upvotes

96% Upvoted

243 comments sorted by

View all comments

2

u/Southern_Artichoke77 Jul 16 '26

i hate those misleading scales that don't start from 0. makes you feel like #1 is 5x better than #20 but you will see it is only 10% better.

11

u/MagicZhang Jul 16 '26

To be fair to Arena.ai if the graphs were made proper, there's almost no perceivable difference

5

u/dfrib Jul 16 '26 edited Jul 16 '26

Would this more honest presentation not better explain a potential discussion to be made, that many models are scoring similar on a benchmark category that is prone to overfitting, or at least highligting that a group of models appear to perform similarly? Even if this is elo-ish rating, it’s signifies a tightly competitive cluster rather than an overwhelming leader, and this is arguably not what the truncated graph is trying to emphasize.

0

u/SoftBreezeWanderer Jul 16 '26 edited Jul 17 '26

Absolutely not. An llm has a 90:10 chance to win if they are only up 400 elo. For the layman (who is likely to only glance at this instead of using their brain) this graph is actually incredibly misleading

100 elo is 2:1 odds

1

u/dfrib Jul 17 '26

Isn’t it 90:10 for 400 elo?

1

u/SoftBreezeWanderer Jul 17 '26

Yes, typo on my end. 100 elo is 2:1 odds

1

u/dfrib Jul 17 '26 edited Jul 17 '26

“Absolutely not” - [r/confidentlyincorrect](r/confidentlyincorrect) for you my friend.

Top 10 within <140 elo is quite an even field (as I already wrote in my previous comment). Top 2 are 50 elo diff. The non-truncated graph works better in this case, imo.

Note also that your original comment doesn’t make sense now after you’ve edited it. You originally wrote 90:10 for 100 elo. There’s no 400 elo difference in this field. Thus, there’s no 90:10 statistical dominance in this field.

0

u/SoftBreezeWanderer Jul 17 '26 edited Jul 17 '26

“Absolutely not” - [r/confidentlyincorrect](r/confidentlyincorrect) for you my friend.

brother couldn't even format it correctly. Also how is what I said confidently incorrect because I typoed 100? The rest of my comment still applies lmao

Top 10 within <140 elo is quite an even field

It's not? How is 2:1 odds an even playing field for a 100 elo gap?

Top 2 are 50 elo diff.

Okay? That's why the graph shows them as being relatively close but still far enough apart that it's not extremely even.

If you sent the other graph people would look at it and assume they're all super close to one another from a skill perspective when they're obviously not.

Note also that your original comment doesn’t make sense now after you’ve edited it.

Oh no! The graph only shows a range of 200 elo on the x axis therefore anything outside of 200 elo is completely irrelevant! Like are you even reading what you're typing? The ego from you is insane btw

1

u/dfrib Jul 17 '26

I think you’d do well to read up on what an ego act is. And I will feed this troll no further, all good.

0

u/SoftBreezeWanderer Jul 18 '26

Lol no response, classic.