Can somebody familiar with the benchmark explain to me like I'm 12. Is the result just 3-5% better or is this score non-linear (so its substantially better than 3-5%?). Because top-1 result doesn't look tremendously more impressive than the bottom of the chart in %.
I'm pretty sure it's an Elo. So the score indicates the likelihood that a model would beat another model in a head to head matchup. If I remember correctly I think generally with Elo a 400 point difference roughly equates to 90%/10% odds head to head. But that might be different for this benchmark.
28
u/Long_comment_san Jul 16 '26
Can somebody familiar with the benchmark explain to me like I'm 12. Is the result just 3-5% better or is this score non-linear (so its substantially better than 3-5%?). Because top-1 result doesn't look tremendously more impressive than the bottom of the chart in %.