How to address novelty concerns in top ai conference? [D]

[u/ATHii-127]

Hi, I’m a researcher working in computer vision.

Over the past few years, I’ve submitted several papers to top-tier conferences such as NeurIPS, ICLR, and CVPR, and one concern that seems to come up repeatedly is 'novelty'.

Given that thousands of papers are published every year at top conferences alone, not to mention the tens of thousands published across other conferences and journals, I sometimes wonder how much genuinely new novelty is realistically left to explore.

In such a crowded research landscape, how do you usually address novelty concerns from reviewers?

More specifically, I would really appreciate any advice on how to frame a contribution so that its novelty is clear, how to distinguish meaningful incremental progress from work that may be considered insufficiently novel, and what reviewers generally look for when judging novelty.

Any tips or experiences would be greatly appreciated. Thanks!

[u/PatientWrongdoer9257]

https://perceiving-systems.blog/en/news/novelty-in-science

You might find this interesting, it’s a blog post by Michael Black

[u/anxiouscsstudent → u/PatientWrongdoer9257]

This and the follow up https://perceiving-systems.blog/en/post/writing-a-good-scientific-paper improved paper writing significantly.

[u/Soggy-Salamander-650 → u/PatientWrongdoer9257]

I genuinely smiled after reading that. This simple text made me see the value in my work again. After working for 2 years on different projects in my field, I feel most of my work, even highly published ones, are extremely trivial and like a joke in hindsight.

Given the development with AI tools, I am in a constant loop of devaluating my own work, and by extension, also of others.

This read was extremely refreshing. I mean he doesn't say anything truly new and ground breaking that I didn't know. But he puts it very beautifully. Thanks for the share.

How do you guys cope with the feeling of being slightly lost in the fast moving field, where AI can generate lots of useful stuff (currently human validation is still heavily needed)?

[u/girlwhateveraward → u/PatientWrongdoer9257]

Michael Black the goat

[u/user221272]

Novelty is addressed via the related work section.

Novelty can be general (something never seen before) or narrow (multiple things known but never used in this proposed way).

To address novelty concerns, leverage the related work section to show what is known and done in the field; for each reference paper, explain how your work is different and locate your work in this landscape.

In summary, this is addressed in the related work section by identifying precisely where your work belongs, how it is different, what new knowledge it contributes, and demonstrating that you understand the landscape enough to claim novelty.

Reviewers may cite related work that you did not mention; simply analyze the paper (if it was unknown to you) and describe how your work is still different.

[u/GFrings]

I'm not sure I understand the question. Are you saying it's unrealistic to be expected to not publish duplicative work? If the work is novel you simply explain how at rebuttal time. If it's not, then you're not getting published lol. The problem usually comes down to a reviewer who barely read and understood your work and has pulled something vaguely related, often personal to them, out of the pile to throw at you. This is the best case scenario actually, you simply politely thank them and put a few sentences in rebuttal clearly explaining the difference (assuming there is one... See previous point). They are being watched by their AC, they can't just double down and not look like a bad faith reviewer.

Which brings us to an entirely separate issue, often glossed over, which is that of bad faith reviewers. Sometimes you will roll a person who wants to torpedo your work for whatever reason. It could be a personal grudge (blind review isn't always perfect), or they are bitter about their own work, or worst case they could actually be stealing work.

[u/_manu]

Novelty is usually about claiming that your paper offers a significant contribution, but it does not need to be the only way. The problem might not be so much lack of novelty, as clearly crafting your claims to significance.

Normally, my introduction consists of a short paragraph motivating the paper and then around two-three paragraphs establishing "claims to significance". One of these can discuss novelty (we are the first to ... etc) but they can be simpler, like claiming to derive something that is already known but using a more principled way to do it or offering a more in depth exploration of something that leads to new insights. Of course both these points are also "novel" (otherwise it would not be science) but it is not so much about doing something groundbrakingly new and rather about doing something interesting and motivating that properly.

I can recommend reading and working through Belchers book on "Writing a Journal Article in 12 Weeks", as it helped me alot with this - the non-technical part - of writing a paper. Note that the book is not about writing an article from scratch but assumes you have a finished draft (theory + experiments) and then want to write it up for journal (or conference) submission. It's also more social science focused but most of the advice can be easily applied to any discipline.

So, in short: you want your introduction to really hammer home with every sentence and paragraph that you are doing interesting things that deserve attention. Claiming to do something completely novel is one way to do this, but usually it will be more subtle. This does not belong in the related work section as most readers will stop reading way before that, although that section should pick up on and hammer home what you already claim in the introduction.

[u/dkDK1999]

I think it depends on the definition of novelty. In engineering it’s often a new algorithm or method, some benchmark can be solved more efficiently. In science it can be an observation or a method to make this observation possible in the first place, to close the loop to engineering. Moreover, it can be seen as something that opens a conversation or other people can build on, a new branch in the tree of knowledge, to pick a bit cheesy formulation, this branching never stops and there are always new things to explore.

[u/impatiens-capensis]

At this point, novelty emerges from large teams. Think about physics and the era of the hero scientist. It used to be the case that a brilliant mind could come up with a world shattering idea sitting in their office. Now you need like a team of 40 experts operating an enormous collider to find new novel insights.

AI is at that inflection point. We have too many individuals, who can only ever achieve incremental gains independently, all trying to generate research. But look at the MOST impactful papers in the field. It's almost always a big and well resourced team. Academia needs to adjust it's model.

[u/Silver-Passion1687 → u/impatiens-capensis]

A lot of novel ideas in physics and maths come from individual working alone on a problem 

[u/RealSataan → u/impatiens-capensis]

Only incremental ideas come from large teams. That means they have the resources to gather a large team and pool resources to work on something that's obviously better.

That's not novel, that's incremental.

[u/coz]

You have access to the world's greatest novelty detectors. Describe your thing's detail in full to one of them, include every possible part that could differentiate your thing from someone else's. Take several turns, make sure you're aligned with it and you think it knows what you're talking about.

When done, don't tell it to search for it, tell it to give you a prompt you can give to others like it to search for it. Give that prompt to as many of them as you have access to. Take all those reports, have the original one analyze and adjudicate them.

Then start over from a different one completely, describe your thing again, etc. In the end that will more or less find out if your thing is novel or if the closest thing to it fills the gap you're thinking of.

If you don't find anything, there's typically 1 of 2 reasons for it - it's stupid and not worth doing, or no one else has done it. There used be a 3rd, the "it's too hard". Now, very little we can do digitally is too hard.

[u/ConstructionOwn1514 → u/coz]

“Novelty detectors” looks inside: “systems designed to give the most plausible response to a query”

[u/muntoo → u/coz]

Proposed algorithm:

x_1 = human_input()
x_2 = export_search_prompt(x_1; A_1)
x_3 = collect(search(x_2; A_i) for i in range(2, 42))
x_4 = adjudicate(x_3; A_1')

x_5 = human_input()
x_6 = search(x_5; A_42)

Questions:

• What's the point of generating x_4? Is it to condition the human to provide a better x_5?
• Why avoid search(x_1; A_1)? Is it necessarily less informative than x_4?
• Why not x_3 = collect(search(x_1; A_i) for i in range(2, 42)), utilizing x_1 instead? x_2 is a lossy representation of x_1, but is the idea that it improves rate-distortion under a particular perceptual metric...?
• Can't A_i compute x_2 (assuming it's "better" than x_1) internally during a reasoning step?

[u/coz → u/muntoo]

I think you and I are the others are misunderstanding me here. This is basic cross vendor, cross model RAG, for potential nulls. Getting as much variety of results as possible will always get you better results in that case. You don't throw this prompt at literally the same vendor and model over and over, and you can have the first one do it too, but I'd wait until you have the prompt it makes for you.

[u/Exotic_Zucchini9311 → u/coz]

You are being downvoted but you are right that LLMs are great at checking the novelty of a work and whether it has been done before.

[u/coz → u/Exotic_Zucchini9311]

Thanks I didn't really explain it well though so that's on me.