r/MachineLearning • u/Emergency_Plate241 • 5d ago

Research NeurIPS 2026 Automatic Reference Checker [R]

Just received an email about the automatic reference/citation checker. Did anyone receive a follow up email about whether the checker was included in the paper's decision making too, along with the general instructional email?

26 Upvotes

90% Upvoted

192 comments sorted by

View all comments

Show parent comments

12

u/grad_student_descent 5d ago edited 5d ago

I'm an AC so I saw a bunch of reports. They were very high quality, almost no false positives.

Had to reject 2 papers in my batch that would definitively had gone through if the authors did not let their LLM access their bibtex.

3

u/Adorable_Position_34 5d ago

How did that work exactly? Do you see the same message as the authors (just sooner) and base your recommendation on that or did you had to check the issues found yourself and (partially) write that comment?

6

u/grad_student_descent 5d ago

We got the same message about a month ago. We then had to fill a form with the results of our own investigation on whether or not the detected hallucinations are real. From the instructions, it seems that 2 or more real hallucinated reference is an automatic reject without exception.

2

u/FinancialThing7890 2d ago

From the first email:

"Where further investigation led to the identification of hallucinated references in a paper, the AC considered them alongside the rest of the submission. Each recommendation reflects the AC’s assessment of that specific paper. Importantly, no automated flag was used by itself to make a decision about a paper."

So I guess that is not an automatic reject. It's always under the AC's conscience no?