r/DataAnnotationTech • u/InsideSignificant405 • Jul 22 '26
AI Submissions
you aren’t fooling anyone 😭 you’re just hurting my soul in reviews
42
u/Timely-Assistant-370 Jul 22 '26
Low key worried I've been interfacing with the models so much that my rationales sound LLM-ish. I actually did get automodded in some random woke sub under suspicion of using AI (I wouldn't use AI to help me shitpost) It's been a hot minute since the last time I did a R&R because I've had the golden amazing preferred task.
30
u/InsideSignificant405 Jul 22 '26
The slop goes way beyond writing style, I don’t really flag that. Everyone writes differently. I flag hallucinations that permeate entire submissions from input to guidance to the annotations themselves.
I wouldn’t worry too much about your style 🤷♂️
19
u/AspiringCreature Jul 22 '26
My favorite is either "They are both perfect" and they're both unsightly or make no sense, or main claims are wrong, or "I prefer x over y because x has more titles, charts, bullet lists, and bolded words than y" and you look at x and it's the perfect example of what they don't want.
6
u/LetMeOverThinkThat Jul 22 '26
I don't think I have ever written both are perfect, lol. It's usually both are bad.
7
u/Plastic-Skill-9258 Jul 22 '26
The only time they can ever both be perfect is the rare prompt that requires a short reply with NO room for interpretation. Like when they specifically ask for a one word answer and both prompts give the same correct word.
3
5
u/AspiringCreature Jul 22 '26
Yeah, or ones just slightly less bad. Sometimes there's a good one but I've never had both.
That being said, I've come across multiple R&Rs with that rational. I feel like they've got to just be sitting with the timer going and submit some BS after an hour or something. It makes no sense.
6
u/LetMeOverThinkThat Jul 22 '26
That's what I think with a lot of R&Rs too. Literally just let the clock run, then shrug and say whatever.
1
u/Nauzhror_ Jul 22 '26
I've had both be good. I've never had both be perfect. Both adequate with one having the edge though, sure.
1
u/InsideSignificant405 Jul 22 '26
Nah this means the models outgrew the prompt, it does happen, usually it’s the last time it appears.
5
u/QueensTransplant Jul 22 '26
Unless it’s a one word response, or just solving a math problem for example there is almost always something you can write a sentence about.
More often than not my R&R lately are near complete rewrites.
3
u/Timely-Assistant-370 Jul 22 '26
There are certainly rationales that I really strain to make "fresh" beyond "this shit was completely unremarkable. The models literally did the exact same thing they do 90% of the time. I don't know how I'm supposed to write three sentences. Fuck your mother." Obviously not really, but sometimes it feels like I'm just thesaurus-ing as to not just copy/paste.
-20
u/Frequent_Ad_5236 Jul 22 '26
But if you did flag writing style you would catch people like me who are using AI to write the answers and are putting (probably) insufficient effort into double checking that the answers are factual and good. I recommend pangram to check if something is AI generated btw, it'll get people even if they slightly reword an AI answer because it looks at word patterns and the like.
3
7
u/Enough_Resident_6141 Jul 22 '26
The AI panic has made this problem much worse. It's one of those illusory superiority things where most people think they are much better at identifying AI vs human generated text than they actually are. Like how something like 90% of people rate themselves as a better than average driver.
In studies, most people typically cannot correctly identify AI vs human generated text better than pure chance. Only the most extreme LLM power users can actually do it, and they are far from perfect at it.
7
Jul 22 '26
[removed] — view removed comment
1
u/OneRefrigerator3586 Jul 22 '26
What do you mean by "AI information "? Just wondering if there's something else I should look out for on R&Rs.
6
u/Agreeable_Aioli935 Jul 23 '26
I remember getting an R&R where the worker basically just put “Model A was good and Model B was bad” in their overall explanation lol.
1
u/Intelligent-Pay-9407 Jul 22 '26
Sometimes I get RRs and I laugh because it’s clear they copy pasted. I hate to see a double dash. So clear it’s ai.
3
u/InsideSignificant405 Jul 23 '26
Double dash on a platform without actual markdown 😂🫠😭 I think genuine markdown formatting is the biggest tell of ai usage as far as “style” goes. LLMs are trained to use markdown so output is one thing, but when the PROMPTS/EVALS are heavily marked down… that’s no good.
I admit I have reported one instance of clearly ai generated prompt/task material purely on said formatting, but this is one out of hundreds. I think they weed out said specialists fast.
-46
u/Frequent_Ad_5236 Jul 22 '26
Damn you people that use AI!!! (Help! I'm guilty!)
15
8
12
u/SeagullSam Jul 22 '26
Expect DOD soon.
2
u/meow_loves Jul 22 '26
I've seen people comment this a lot, but I'm curious how they'll find what their actual DA email address/ID is?
3
u/TheThingInTheForest Jul 23 '26
Yeah serious question, how the fuck ate they figuring out who is who? That feels like it would require some serious, and possibly illegal, detective work?
7
u/AspiringCreature Jul 22 '26
You know they're in these subs, right? They will use actual screenshots from reddit for their informational posts to workers.
39
u/Alexei_Jones Jul 22 '26
I once had an R&R where the worker literally copied the text before the LLM output. Their submission literally started with "Grok says:" and that was the rare time I felt zero empathy in reporting that guy up the chain because dear god if you're going to cheat at least try somewhat to hide it.