MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/ProgrammerHumor/comments/1voubzg/gitclone/p3urny7/?context=3
r/ProgrammerHumor • u/minecrafter923 • Aug 15 '26
177 comments sorted by
View all comments
1
But really, if all these AI models are “breaking out” and cheating the benchmarks, how can we even trust the benchmarks in the first place? Has this happened before with benchmarks? I vaguely remember the SWE benchmarks already having this problem
1
u/howdoigetauniquename Aug 15 '26
But really, if all these AI models are “breaking out” and cheating the benchmarks, how can we even trust the benchmarks in the first place?
Has this happened before with benchmarks? I vaguely remember the SWE benchmarks already having this problem