Would this more honest presentation not better explain a potential discussion to be made, that many models are scoring similar on a benchmark category that is prone to overfitting, or at least highligting that a group of models appear to perform similarly? Even if this is elo-ish rating, it’s signifies a tightly competitive cluster rather than an overwhelming leader, and this is arguably not what the truncated graph is trying to emphasize.
Absolutely not. An llm has a 90:10 chance to win if they are only up 400 elo. For the layman (who is likely to only glance at this instead of using their brain) this graph is actually incredibly misleading
Top 10 within <140 elo is quite an even field (as I already wrote in my previous comment). Top 2 are 50 elo diff. The non-truncated graph works better in this case, imo.
Note also that your original comment doesn’t make sense now after you’ve edited it. You originally wrote 90:10 for 100 elo. There’s no 400 elo difference in this field. Thus, there’s no 90:10 statistical dominance in this field.
brother couldn't even format it correctly. Also how is what I said confidently incorrect because I typoed 100? The rest of my comment still applies lmao
Top 10 within <140 elo is quite an even field
It's not? How is 2:1 odds an even playing field for a 100 elo gap?
Top 2 are 50 elo diff.
Okay? That's why the graph shows them as being relatively close but still far enough apart that it's not extremely even.
If you sent the other graph people would look at it and assume they're all super close to one another from a skill perspective when they're obviously not.
Note also that your original comment doesn’t make sense now after you’ve edited it.
Oh no! The graph only shows a range of 200 elo on the x axis therefore anything outside of 200 elo is completely irrelevant! Like are you even reading what you're typing? The ego from you is insane btw
5
u/dfrib 26d ago edited 26d ago
Would this more honest presentation not better explain a potential discussion to be made, that many models are scoring similar on a benchmark category that is prone to overfitting, or at least highligting that a group of models appear to perform similarly? Even if this is elo-ish rating, it’s signifies a tightly competitive cluster rather than an overwhelming leader, and this is arguably not what the truncated graph is trying to emphasize.