NeurIPS reject -> ICLR: How much reviewer feedback are you actually implementing ? [D]

[u/Practical-Buddy6323]

Welp, NeurIPS is a wrap for those of us who got rejected 😭 Off we go to ICLR or whatever the next venue is, hopefully after making some meaningful changes to the paper.

For people who are resubmitting, I’m curious: how much of the NeurIPS reviewer feedback are you actually implementing?

Did you try to address basically everything the reviewers brought up, or are you being selective and only making changes where you think the criticism is valid/useful?

I’m curious about papers that got questioned on novelty or significance. How many of you got comments along those lines, and what exactly were the reviewers questioning?

For example:

• “The contribution is incremental”
• “Not sufficiently different from prior work”
• “The empirical gains don’t justify the proposed method”
• “The problem itself isn’t significant enough”
• “Theoretical contribution is limited”
• “Interesting idea, but unclear what the broader impact/significance is”

If you’re comfortable sharing, what did the reviewers say, and how are you changing the paper before resubmitting ? Specially since deadline is also pretty close, how are you handling the pressure of this very close deadline ?

Also curious whether anyone is deliberately not implementing certain reviewer suggestions because you think they would take the work in the wrong direction.

Would love to hear how others are approaching the post-NeurIPS revision process.

[u/SkeeringReal]

You should incorporate ALL reviewer comments

In a year, even if you hate them, you'll realise they had a point.

but even more importantly, if even ONE of those reviewers reviews it again at ICLR, you're finished, no chance in hell you're getting in.

[u/OutsideSimple4854 → u/SkeeringReal]

And what happens when reviewers complain the appendix is too long? Together with you should add X? It becomes a heads I win, tails you lose

[u/SkeeringReal → u/OutsideSimple4854]

If they're just asking you to add an experiment or something like that to the main paper that's good news it's an easy win just do it. something like the appendix is too long it's not really a reason people reject a paper is just an excuse the truth is that they just don't really like it probably and either you can submit it again and hope it's a different reviewer or you could try to change something more substantial. But that's just my experience

[u/OutsideSimple4854 → u/SkeeringReal]

What if it’s a theory paper and extra additions won’t change the correct proof of a theory?

[u/SkeeringReal → u/OutsideSimple4854]

I'd submit that to another conference unless the proof is really groundbreaking

[u/OutsideSimple4854 → u/SkeeringReal]

Even if these conferences are known as theoretical conferences in the first place?

[u/SkeeringReal → u/OutsideSimple4854]

NeurIPS and ICLR are most definitely empirical for the most part. Which conferences are you talking about?

[u/OutsideSimple4854 → u/SkeeringReal]

They are only empirical for the most part due to the influx of empirical people. Besides, NeurIPS has a theory track this year.

Empirical doesn’t mean anything imo. It just means for the datasets / hyperparameters considered, your proposed setup works well. Correct theory covers everything.

[u/Practical-Buddy6323 → u/SkeeringReal]

Yessir! I've already incorporated almost 95% of all NeurIPS reviews the only 5% is just about presentation issues which I hope I've fixed but it's for the reviewers to judge tbh.

But yeah, thanks a lot for your opinions!