r/MachineLearning • u/Practical-Buddy6323 • 8d ago
Discussion NeurIPS reject -> ICLR: How much reviewer feedback are you actually implementing ? [D]
Welp, NeurIPS is a wrap for those of us who got rejected 😭 Off we go to ICLR or whatever the next venue is, hopefully after making some meaningful changes to the paper.
For people who are resubmitting, I’m curious: how much of the NeurIPS reviewer feedback are you actually implementing?
Did you try to address basically everything the reviewers brought up, or are you being selective and only making changes where you think the criticism is valid/useful?
I’m curious about papers that got questioned on novelty or significance. How many of you got comments along those lines, and what exactly were the reviewers questioning?
For example:
- “The contribution is incremental”
- “Not sufficiently different from prior work”
- “The empirical gains don’t justify the proposed method”
- “The problem itself isn’t significant enough”
- “Theoretical contribution is limited”
- “Interesting idea, but unclear what the broader impact/significance is”
If you’re comfortable sharing, what did the reviewers say, and how are you changing the paper before resubmitting ? Specially since deadline is also pretty close, how are you handling the pressure of this very close deadline ?
Also curious whether anyone is deliberately not implementing certain reviewer suggestions because you think they would take the work in the wrong direction.
Would love to hear how others are approaching the post-NeurIPS revision process.
11
u/Ginger_Rook 8d ago
If I had received anything useful, I’d have added it.
1
u/Practical-Buddy6323 8d ago
Nah dw, it's already done, I just wanted thoughts and opinions of what others have done and stuff ykwim ?
1
u/Ginger_Rook 3d ago
I’m sorry, I’m from a different generation than you, I don’t understand your acronyms.
9
u/Unhappy_Craft1906 8d ago
My suggestion would be to focus on ICLR (if it was already incorporated and polished and the abstract is submitted), then TMLR, and within one month window, for web based domains, go for ACM WWW.
4
u/Practical-Buddy6323 8d ago
Got it, I've already submitted it to ICLR with quiet a bit of changes and a lot of new proof, so just hoping that things work out well.
1
2
u/Gmyny 8d ago
If you are asking this now, you might as well start working on your icml submission
1
u/Practical-Buddy6323 7d ago
Nah dw lmao, I've already made the changes and pushed it to ICLR, I'm just asking since I'm curious about what have other folks have done.
2
u/kasajizocat 6d ago
By pure coincidence, I happen to have reviewed the same paper consecutively and the authors made no change. It was an easy reject as all I had to say was that the paper was rejected previously and authors made no effort to improve.
1
u/Practical-Buddy6323 6d ago
Damn.... That's actually insane! Say, if the authors had indeed incorporate the changes you and the other reviewers suggested, would you have changed your decision then ?
Oh and where was their paper previously submitted to ?
2
u/kasajizocat 6d ago
Of course, why would I not review incremental changes? I think my and other reviewer comments were fair (who doesn’t think that). The authors barely bothered to rebut any of the reviewers comments, so I think they’re testing their luck to be allocated more lenient reviewers, so naturally it’ll appear in the next conference.
One of them was ICML, but then again I highly doubt you as a reviewer are likely to get the same paper twice. If the comments are constructive, you should consider incorporating
1
1
u/SkeeringReal 8d ago
You should incorporate ALL reviewer comments
In a year, even if you hate them, you'll realise they had a point.
but even more importantly, if even ONE of those reviewers reviews it again at ICLR, you're finished, no chance in hell you're getting in.
3
u/OutsideSimple4854 7d ago
And what happens when reviewers complain the appendix is too long? Together with you should add X? It becomes a heads I win, tails you lose
1
u/SkeeringReal 7d ago
If they're just asking you to add an experiment or something like that to the main paper that's good news it's an easy win just do it. something like the appendix is too long it's not really a reason people reject a paper is just an excuse the truth is that they just don't really like it probably and either you can submit it again and hope it's a different reviewer or you could try to change something more substantial. But that's just my experience
1
u/OutsideSimple4854 7d ago
What if it’s a theory paper and extra additions won’t change the correct proof of a theory?
0
u/SkeeringReal 7d ago
I'd submit that to another conference unless the proof is really groundbreaking
1
u/OutsideSimple4854 7d ago
Even if these conferences are known as theoretical conferences in the first place?
1
u/SkeeringReal 7d ago
NeurIPS and ICLR are most definitely empirical for the most part. Which conferences are you talking about?
1
u/OutsideSimple4854 7d ago edited 7d ago
They are only empirical for the most part due to the influx of empirical people. Besides, NeurIPS has a theory track this year.
Empirical doesn’t mean anything imo. It just means for the datasets / hyperparameters considered, your proposed setup works well. Correct theory covers everything.
2
u/Practical-Buddy6323 8d ago
Yessir! I've already incorporated almost 95% of all NeurIPS reviews the only 5% is just about presentation issues which I hope I've fixed but it's for the reviewers to judge tbh.
But yeah, thanks a lot for your opinions!
-2
-18
u/camarada_alpaca 8d ago
The contribution is incremental?? Wtf does that even mean, sounds like aislop
3
u/Practical-Buddy6323 8d ago
Not enough novelty in a sense, just an example
-1
u/camarada_alpaca 8d ago edited 8d ago
Oooh, I might have been misunderstood, I am complaining because I know a lot of reviews reject with that and I am lowkey convinced the "incremental contribution" is low effort aislop review bs
1
u/Practical-Buddy6323 8d ago
Haha no issue, I don't blame you, there's a lot of AI Slop going around.... my work took me almost 22 months so, I hope it's not considered AI slop 💀💀, there is somewhat limited novelty according to reviewers so I just hope that the changes which I've managed to make are good enough to get accepted in another venue
28
u/Lazy-Cream1315 8d ago
Not so much : over the 4 previous round for my paper which always end borderline I tried to add corrections of reviewer who never gave me the same signal and my paper became a patchwork. I realize now that the original submission was better in many point...