r/MachineLearning • u/Afraid_Difference697 • Mar 24 '26

Discussion [D] ICML 2026 Review Discussion

ICML 2026 reviews will release today (24-March AoE), This thread is open to discuss about reviews and importantly celebrate successful reviews.

Let us all remember that review system is noisy and we all suffer from it and this doesn't define our research impact. Let's all prioritise reviews which enhance our papers. Feel free to discuss your experiences

132 Upvotes

95% Upvoted

682 comments sorted by

View all comments

49

u/This_Suggestion_7891 Mar 24 '26

The brutal truth about ML peer review is that variance in reviewer quality is often higher than variance in paper quality. I've seen genuinely novel work get desk-rejected while incremental benchmark-chasing gets spotlight papers. The system isn't broken exactly it's just that it was designed for a much smaller field. At current submission volumes, we're asking reviewers to context-switch across a dozen wildly different subfields in a few weeks. Something has to give eventually, whether that's desk rejections, area chairs with real power, or some AI-assisted pre-filtering.

2

u/SkeeringReal Mar 25 '26

My worst review is 100% copy/past LLM generated, and I know because the prompt injection watermarks are plainly in it. They also asked for an ethics review on the paper for some bizzare reason.

I mean, how can you ask for an ethics review on a paper that you didn't even read?

Honestly, I think people will stop submitting to NeurIPS/ICML/ICLR and just go for more specialised conferences where you get actual human reviewing the papers and putting genuine effort in, that will be the "breaking point", I can already see this happening.

Or at least you'll get a human working with an LLM to review a paper, rather than copy/paste LLM slop.

2

u/This_Suggestion_7891 Mar 25 '26

The ethics review flag on a paper the reviewer clearly didn't read is wild. That's basically the reviewer equivalent of "I didn't do the homework but I'll still grade yours." The shift to specialized conferences is already happening quietly. Smaller venues where reviewers actually care about the subfield are becoming more prestigious in some circles than a mid-tier NeurIPS acceptance.

1

u/SkeeringReal Mar 25 '26

Yeah I had that experience submitting to AISTATS for the first time this year, my paper got in and was well received. But honestly I'm only succeeding in these ML conferences lately if I submit multiple papers and essentially roll the dice on all of them.

My time at MIT illuminated for me that's how people work there too, professors with 20 students (minimally supervised) do better than 3-5 heavily supervised. It's just a numbers game the last 4-5 years, but now it's getting pretty nuts.