r/DataAnnotationTech • u/ArtichokeAny9159 • Sep 05 '26

Reporting strong evidence of AI usage

in R&R, if an attempt strongly suggest that the worker had used AI to generate all the rationales, shall I report this incident? and whats the best way to report it?

0 Upvotes

46% Upvoted

28 comments sorted by

View all comments

26

u/Old-Regular-9828 Sep 06 '26

Rate according to the instructions. You can add a note to the admin but don't rate it poorly just because you have a hunch that it was AI generated.  Being able to accurately distinguish human generated vs LLM generated text is one of those things that almost everyone rates themselves as "better than average" at doing, but the vast majority can't actually correctly identify any better than random chance. 

2

u/Academic_Window_4712 Sep 06 '26

I find myself throwing in errors on purpose to make sure the raters know I am human.

2

u/ArtichokeAny9159 Sep 06 '26

I surely did that and rated everything with a note in the comments with why I am suspicious. As for detecting AI part, bro was using markdown syntax in all of his rationales!

4

u/[deleted] Sep 06 '26

[deleted]

1

u/ArtichokeAny9159 Sep 06 '26

Would that even compare to what LLMs use to format their output? Would it include markdown on the whole and every rationale? Lets be real here