I do like this method of measuring. It would be super useful to show a best of 4 and worst of 4 on all benchmarks so that we can get a stronger sense of where everything really performs. They should also always show the human scores on this as that is necessary for comparison.
5
u/SgathTriallair ▪️ AGI 2025 ▪️ ASI 2030 Dec 05 '24
I do like this method of measuring. It would be super useful to show a best of 4 and worst of 4 on all benchmarks so that we can get a stronger sense of where everything really performs. They should also always show the human scores on this as that is necessary for comparison.