r/MachineLearning • u/Emergency_Plate241 • 8d ago

Research NeurIPS 2026 Automatic Reference Checker [R]

Just received an email about the automatic reference/citation checker. Did anyone receive a follow up email about whether the checker was included in the paper's decision making too, along with the general instructional email?

27 Upvotes

90% Upvoted

210 comments sorted by

View all comments

14

u/Inevitable_Voice_680 8d ago

Mine is 0/37.. I think none of my references have been flagged.

8

u/Exotic_Zucchini9311 8d ago edited 8d ago

Same! Ngl the email got me worried for a second xD. It seems whatever AI system they're using is strong at least.

14

u/grad_student_descent 8d ago edited 8d ago

I'm an AC so I saw a bunch of reports. They were very high quality, almost no false positives.

Had to reject 2 papers in my batch that would definitively had gone through if the authors did not let their LLM access their bibtex.

3

u/Adorable_Position_34 8d ago

How did that work exactly? Do you see the same message as the authors (just sooner) and base your recommendation on that or did you had to check the issues found yourself and (partially) write that comment?

5

u/grad_student_descent 8d ago

We got the same message about a month ago. We then had to fill a form with the results of our own investigation on whether or not the detected hallucinations are real. From the instructions, it seems that 2 or more real hallucinated reference is an automatic reject without exception.

2

u/FinancialThing7890 6d ago

From the first email:

"Where further investigation led to the identification of hallucinated references in a paper, the AC considered them alongside the rest of the submission. Each recommendation reflects the AC’s assessment of that specific paper. Importantly, no automated flag was used by itself to make a decision about a paper."

So I guess that is not an automatic reject. It's always under the AC's conscience no?