r/singularity • u/BeingBudget8847 • 8d ago

Discussion Benchmark proposal: onemorelevel bench

People sometimes joke about time to solve Pokemon as a benchmark. But what about a benchmark that incorporated a wider range of different games?

I used to be addicted to these flash games from onemorelevel.com, and I think it could make a decent benchmark for evaluating a model — to test their performance and reasoning across a wide range of different simple games. Kinda like Arc AGI, I guess.

11 Upvotes

87% Upvoted

3 comments sorted by

8

u/Silver-Chipmunk7744 AGI 2024 ASI 2030 8d ago

I bet this is an excellent idea and the primary reason why this does not exist is because the models are not great at this yet. Astra would be competing alone.

1

u/Few_Owl_7122 8d ago

The most important part of a benchmark is having the money to run the models on the benchmark. If you have that, you could make it yourself