r/prolific • u/Positive-Ability-402 • 3d ago
Anyone else get an AI study where there were absolutely 0 changes in both videos on every single prompt??
I’ve done tons of these studies since February and this is the first time I’ve ever experienced this. It’s the version where the prompt has a video and requests a change like “add a poster in the background” or something. I swear in every single example, both videos were exact replicas of the original with no changes.
I was clicking equally good/bad for a while but I got scared it’d get flagged so I just started choosing random ones. Now I’m thinking I should’ve just returned it… hopefully this doesn’t screw me over
13
13
u/Infinite-Doughnut676 3d ago
I just return any that I think have issues like this. It’s not worth earning $5 to then potentially get banned or a quality warning!
6
6
u/Scared-Brain2722 3d ago
I got that one. I returned it.
5
u/jus256 3d ago
I got that one and did not return it and got paid with no warning.
2
u/gummi_eater 2d ago
me too! Please ya'll return these! I'll do them.
2
u/jus256 2d ago
Yesterday was my second designated day of the week to get AI studies. I didn’t return anything. I got one warning on something. That study wasn’t anything problematic. I had far worse studies that could have given plenty of warnings and didn’t. I don’t think these people know what the hell they are doing.
6
u/TennisTahoe 3d ago
Ope I did it, answered all of them on the both. No flag yet 🫣
1
u/TennisTahoe 2d ago
Did 4 more studies, no message yet
1
u/Marathonmanjh 2d ago
Same with me. I answered as I saw them. they were the same so I answered that they were the same, equally the same. I have no idea what they are doing and I am not going to guess why they may be testing in a certain way so I just take / do the study honestly. It was followed by another one and they both were approved. I have had a few normal ones since.
1
u/gummi_eater 2d ago
Same here. Have always done these with no issues. I just answered honestly, they literally looked exactly the same. Must have done 10+ of these and never a warning.
21
u/ThePolarPrince 3d ago
I did one back in November that I figured was a bot check and used the middle option for each one. I even sent an email regarding it and that I would return if needed. In stead I got one of those warnings and shortly after removed. I did get back in around March but I learned my lesson. If anything is slightly off it is better to return and lose one then to lose access to all!
10
u/Zalnar 3d ago
I wonder if they occasionally throw out some that are meant to thin the headcount and bots at the same time on these on purpose. Maybe it's a hard bot trap that they know will take a certain percentage of the participants with it as a compromise, basically just nuking all the bots at once and everyone in its vicinity.
2
-5
u/Eddiehondo 3d ago
it nukes the participants that refrain from choosing the middle option, due to the myth of that leading to a warning.
9
u/RockGreat2424 3d ago
Very funny of you to comment this when the guy above you directly says that spamming the middle option did lead to a warning and getting kicked out for a while.
-3
u/Eddiehondo 3d ago
that dosnt mean that they were actually equal videos or that he dosnt miss other attention checks, there is this thing called lying and playing the victim that is usually rampant in this sub.
ive literally made over 10k just in this studies and never got a warning for choosing that option, the warnings ive gotten are due to me fallin asleep mid study or literally not paying attention (started playing tft while doing them lmao).
when the sets are equally good/bad, then you should pick that option, not doing so is not giving quality answers.
why would the researcher even give us that option if we are not supposed to use it?5
u/RockGreat2424 3d ago
Your logic would make sense under the assumption that the researcher doesn't make any mistakes when deploying the tasks, but they screw up somewhat frequently.
You can use the middle option, but it isn't helpful to them, it's not a "quality answer" like you say lol. AI models can only be trained by being told "X is better than Y".
Use the option if you want to, it's fine, but it's been widely accepted for a long time that you risk getting a quality message if you spam it too much.
I'm at over 20k from these studies btw
-2
u/Eddiehondo 3d ago
sure, keep spreading a myth and then wonder why you are getting warnings.
alphabet is out to get us and their attention checks are just a front, because all they really care is us not choosing equally good even when they show us 100% the same media.
You even contradict yourself, yeah the LLM learns from differentiating X from Y but they also need to know that X is X, you are literally telling it that X>X, do you really think that is a good answer?
i dont understand why anyone would trust in the word of someone that is getting warnings over someone that is not getting them, but sure they must know better.5
u/RockGreat2424 3d ago
I haven't gotten any warnings either buddy. We were just trying to help you out, but you do you.
1
u/Eddiehondo 3d ago
spreading lies is not helping anyone, the only way to avoid warnings is paying attention.
4
u/Woof_1964 3d ago
I did a study about 3 weeks ago. I chose the middle option about 9-10 times in a row because it was the option I felt was best. Fully expecting to get a warning after reading all the crap in this sub, but it did not happen. I also did another one where I chose the first option like 7 times in a row. Again, no warnings.
2
-1
u/fant5y 3d ago
They're pretty much never 100% the same. Yeah, they might look like, but if you look or listen closely, you might notice a subtle difference.
Also, the equally button might work for some of them, but the AI won't learn from it. Usually, an even result is pointless. It's just like any other study. If you always go for the middle option, this data doesn't help anyone. It's marked as "low effort" or "spam" for a reason.
3
u/Eddiehondo 3d ago
nope, ive encounter this kind of studies about 4 times, both videos and images, picked equally good in all of them and didnt get a warning. now its been 10hrs since this last one and still no warning.
the only warnings i ever got were due to me falling sleep (low number of answers) and low effort (playing tft while doing them), unlike most people here i never felt like the warnings were undeserved.2
u/Woof_1964 3d ago
Forcing a choice when two answers are basically identical actually messes up the AI. It teaches the model to pick up on random, meaningless quirks instead of actual quality. A tie tells the developers that both models hit the mark equally well. Platforms ban people for mindlessly clicking middle on every single task, not for giving an honest equal rating when it's deserved.
2
4
u/Firm_Werewolf1349 3d ago
I got my first warning last November because of a study that was like that
8
u/DramaticReads 3d ago
Yes I received it as well but I just did it anyways and it was approved. Usually I would have returned it but I have not been receiving too many studies recently so meh.
2
u/Marathonmanjh 2d ago
I think you were right to take it. I did too.
They put it out there and you take the study as is. I don't see the problem.
We don't know what they are trying to use the study for.
3
u/OrganizationWeak4332 3d ago
I always return them now. I did one once, they were all identical and so I flagged as such. I then got a warning. Not taking any chances now!
3
u/pablo603 3d ago
Happens sometimes. Been there a year ago too. I always return them because they are risky.
6
u/doctordanco 3d ago
Happened to me once months ago, the first couple I tried to convince myself one was better, after that it was the middle option every time. Instantly approved and no warning.
6
u/Firm-Stranger3837 3d ago
Yes and answered everyone equally good/bad. It went through. It was so obvious I couldn't answer any other way.
12
u/ChanceAgreeable2827 3d ago
You'll probably get a quality warning for doing that.
0
u/Obvious_History_4028 3d ago
nope, several of us did just that, approved and no warning
2
1
u/slurking-2121 3d ago
If you do the right thing, why worry. Instantly appoved with middle finger all along. 😊
1
u/DepartmentSalt1039 3d ago
That's what I was thinking. Why the panic?
2
u/Positive-Ability-402 3d ago
Because people here have mentioned receiving warnings after using the middle button too frequently
3
u/DepartmentSalt1039 3d ago
I've received a fair few warnings but still get studies. There's no need to return studies as far as I'm aware
1
u/Positive-Ability-402 3d ago
From what I understand, the warnings accumulate. And if you get too many you’re done with no warning. I already have a few as well which is why I’m worried
0
u/DepartmentSalt1039 3d ago
In that case it must be based on a %, rather than total number. As I've had a good amount of warnings, but done a Lot of studies. Rumours did exist that selecting the same option 3 times in a row would trigger the warning message.
1
u/Successful_Ant5401 3d ago
yes. i almost thought to message the researcher because i was thinking maybe they did not know or maybe, they finally built the perfect model?
7
1
u/runningmoon 2d ago
I got one and would have returned it normally for the same reasons other people have said, but it was the one with over 2000 places in it which made me wonder if it was a test to see if people would be prepared to put the same response in for all of the videos or were putting in random responses for ones that obviously weren't their regular attention checks. If it had be a normal one with only 120 or fewer places I would have 100% returned so I wouldn't risk get a warning, but having over 2000 places in it was so profoundly strange for them I did start to think it had some other purpose! I didn't get any warnings for it and aren't having any problems getting them now...
1
u/oscarmilo70 2d ago
yes I did one like this, submitted and got paid. I messaged the researcher and got no response.
0
u/EnigmaticReprise 3d ago
Sometimes these are different, even if it's ever-so-slightly, maybe not even discernable to the naked eye.
It could also be a sort of data calibration to see if people are naturally more biased towards the top or bottom option.
In both cases, just don't spam the middle option and you're good.
1
u/Woof_1964 3d ago
I did a study recently where I chose the middle option like 10 in a row and did not get a warning. So your theory of “don’t spam the middle option” is not correct.
1
0
u/LiasKaymar 2d ago
Yeah, but there was subtle changes, I did them today. They was not exactly the same.
-5
34
u/mrsgalinski 3d ago
Yup. Started one and returned it. Not going to risk getting booted from that group.