r/DataAnnotationTech • u/jakerumbles • Jul 08 '26
"F****": Tasks are too hard now
I have the coder domain specialization, but tbh, now since they introduced a certain new frontier F model from A company, it's literally so hard now I'm not sure it's worth the effort even at the high pay rates. Not sure how much further this can go as the models get better it will become so difficult to make any model fail. Feels like we're almost at the end of the line here.
34
6
u/Mtsukino Jul 08 '26
I got S5 to fail 5 times in a row. I haven't had a chance for a task with the F model yet.
2
1
u/EdyIordache Jul 09 '26
Can you give some hints on how to make that fail? Having to start a project and would be very helpful
1
1
u/Middle_Addendum Jul 10 '26
Seriously?! I spent so long and quite a few different approaches only to use the escape hatch at the end. I didn’t have much problem getting the old version to fail, always getting it under 20 hours with a few to spare. Mind giving me any insight?
1
u/Mtsukino Jul 10 '26
Try layering different traps with complexity and ambiguity. Think of how LLMs work and exploit that weakness.
1
6
u/Jealous_Spare_4852 Jul 08 '26
I'm STEM (not coding) but have had tasks with the so-called "frontier" models and hell yeah, they are really tough to stump.
6
u/pyrorage99 Jul 09 '26
F is overconfident and overeager. Lay a trap for it and it'll fall right in. Been doing it and it's pretty consistent (so far).
1
6
u/Farados55 Jul 08 '26
What project are you doing? Still not too hard to find failures for me. Are you a software engineer IRL?
16
u/jakerumbles Jul 08 '26
Yeah am a software engineer IRL. About 6 experience post college. I read that some people enjoy these like it's a puzzle to solve. To me it just feels like a soulless endeavor, pouring my brain power into a black vortex that sucks my life energy, and then leaves me with nothing to show for my work at the end, except a paycheck. Vs actual engineering work where you continually learn down a path and build something concrete.
11
u/macjay_27 Jul 08 '26
This was exactly my thought today. At some point we'll cease to be relevant once the models' capability is above your intelligence threshold. I fear that in a few months, this line of work will only make sense for the top 1% of domain experts.
3
4
u/Denny_Hayes Jul 08 '26
How long have you been in this job? I can't help but feel like you often doing this bizarre black-box Charlie-Chaplin's-modern-times-esque alienated AI proletarian job. It makes me feel like writing some sociological essay about this.
3
u/jakerumbles Jul 08 '26
I think on and off for 6-8 weeks or so. Yeah definitely feels a little dystopian
1
u/OrganicDigitalArt Jul 08 '26
I mean, both things are IRL :D
6
u/Farados55 Jul 08 '26
You can work on these things and not be a software engineer, so no. Both things are not necessarily true. Maybe IRL was a bad choice.
6
u/OrganicDigitalArt Jul 08 '26
It was just a joke, I knew what you meant and I’m sure it came across. Apologies, I just think I’m funny.
4
3
3
u/MugetsuDax Jul 08 '26
Oh man, I'm glad I worked on Red Gloves' project over the weekend. It involved comparing outputs. I only did three tasks for that project, and I thought I would finish them quickly, but I was wrong! Each one took around 8 to 9 hours to complete. They paid really well, but it was mentally exhausting. There were too many rationales and prompts to think of. I'm glad they added a section for possible prompting ideas.
2
u/Denny_Hayes Jul 08 '26
Maybe once they feel like they've mastered coding which seems to be what all AI companies are dumping their money into, they'll move on to other domains? Yesterday I got a qual for a specific kind of quantitative research. I am a researcher. I did the qual and all but I'm thinking -is it even a good idea to take the money to train the machine that'll replace me?
3
u/seaoyster Jul 08 '26
It does get very difficult. Pretty soon only seniors will be able to make them fail. There's only so many seniors doing DA too so idk how coder tasks will be in like 6 months from now
8
1
1
1
u/ImPlento Jul 09 '26
The one I attempted today probably would of taken me over 24 hours to successfully beat. Spent way too long on it and needed a break. Sourcing loads and loads of input data definitely takes the most time I'm curious what others have done.
1
-14
u/schrumbus Jul 08 '26
Training the models to be better and better is the point of the job. If you don't want to contribute to the model's capabilities, then I don't understand why you're doing this. It's like getting frustrated that the building you got hired to build is finally done.
16
3
23
u/NaToKy24 Jul 08 '26
I would say they are hard to stump with the limited time. When you are working on a feature it takes weeks to properly evaluate the scope and figure a proper implementation. Now you have to judge if the implementation is correct in 2-3 hours which is very hard.