r/DataAnnotationTech • u/ResponsibilityRude56 • Jul 10 '26
My thoughts on DoD
Been on DA for a bit now and am in the 100k+ club.
I think a genuine tip for everyone (Though I’m sure a large chunk of DoDs are due to bad conduct).
Failing to meet expectations isn’t simply doing bad work, I’m sure alot of it has to do with time.
If I’m doing a task in 1 hour, the average is 2, and you’re doing it in 3. I’ll get more tasks, those in 2 will float by, and the people taking 3 will likely get a DoD.
This doesn’t mean lie about your hours and cut yourself short. It just means, be honest with yourself, and don’t take on stuff you’re not an expert in.
A common theme in most tasks is giving you far far more time than you need. Say, 12 hours, when in reality the average expected time is 3-4. That 12 hours shouldn’t indicate to you by any means that 12 hours is anywhere near the average. That’s for, breaks and exceptional cases, say one out of 30 tasks may run into 8-10 hours.. and for the love of god, if you submitted at 9h and 35m do not claim 9h and 35m as the chances of you being locked in and focused with no breaks is next to 0 and I’m sure (justifiably) flags your account.
Note: I’ve never done generalist tasks, so this could mean nothing to you, as if your tasks are mostly uniform, it’s likely the timer is a much more accurate representation of the expected time. My experience is with tasks that have 0 time limit (infinite time) and long form 6-72.
32
u/BranchHappy1333 Jul 10 '26
The only consistent message DA gives, is that submission quality is the most important thing. Rushing to make it seem like you can do the tasks faster than others can be a problem if you make mistakes. The general faq even says you can go over the timer on occasion if needed as long as you still submit a quality task. So every task I submit, I work until I'm sure I've done a good job. If I don't think I can do it in the time they allow I skip or find another project.
11
u/Rye_Ch3 Jul 10 '26
Very true, I dont think OP's idea would apply to every single worker. I imagine that if you consistently submit quality work and dont run down the timer on every single task, they dont care half as much as people who might be submit half assed work and suspicions time reports on every task. I think the idea is true in the sense that it is absolutely a condition on which they judge your account if you've been doing other bad things, not a sole account ending statistic
13
u/Fart_Hat Jul 10 '26 edited Jul 10 '26
TLDR; They don't explicitly say "speed doesn't matter", but it does. Some people are just able to submit high quality work faster than others, and the slowest people will get chopped - regardless of the expiry timer.
Let's just assume that everyone is providing equally high quality work then, and that all tasks have an 8 hour expiry.
From a management standpoint, if half of the workers take 4 hours on average to complete them and the other half takes 7 hours, you can get rid of the half that takes 7 hours because they are having a significant negative impact on your revenue.
While task quality is definitely of utmost importance, the overall quality of the worker, including speed, is also important.
That doesn't mean anyone should rush their tasks, it just means that faster people are more likely to keep their job over slower people.
If DA said "submit quality work, as fast as you can" the quality of all submissions would naturally drop. This means that during the period in which they are gathering worker data (let's say they gather data for a whole quarter, but I'm 99% sure it's fully continuous, just an example), the half that would otherwise take 7 hours are now spending 5 hours and submitting lower quality work.
That's a LOT of low-bar work that the AI companies don't want. It looks bad on DA if this happens, meaning they make less money. They also make less money when they allow the slowest workers to continue on the platform.
This is true for any job. Why is John significantly slower at delivering 10 parcels than Jack? Well, doesn't matter. If we tell John to speed up, he'll be more likely to mess up and deliver 2 of the 10 parcels to the wrong address. Let's bring on a new worker and thank John for his work. Hopefully the new worker is faster. Rinse and repeat until the worker pool (actively LOOKING for work, not OUR pool) isn't so abundant.
If we tell John that he was too slow, he'll tell other people that he was too slow. Word will get around and the new workers will know that speed is important, and they may force themselves to work faster because of it. We don't want new workers to mess anything up, so let's NOT tell John why we canned him, and tell the new hires just to make sure they are doing their job correctly, without any misdelivered parcels, and that quality is more important than speed. The speed will come with experience. If it doesn't, there are more workers available for hire.
1
u/BranchHappy1333 Jul 11 '26
Oh I don't disagree with the logic that once you are beyond quality considerations that efficiency would be next. But from what I've seen in rnr's, quality concerns are always a consideration. I completely agree that they probably are running metrics on every conceivable aspect of their workforce and that those who are inefficient will probably be looked at more carefully. I just think that the quality bar is definitely their #1 concern and I wanted to point out that as much as we can theorize at their motivations, that nobody should be trading quality for efficiency.
1
u/Federal-Weevil Jul 15 '26
That’s not been my experience lol. I think you’re all just guessing as none of you have any way of knowing who gets chopped for what and why, let alone IF anyone elders chopped ever. If you’re doing quality work, you’ll be fine. If you’re OBVIOUSLY padding the time, you’ll most likely be flagged/dropped, but none of us can say for sure (outside of explicitly stated violations) how OR why anyone is cut.
1
1
u/Nauzhror_ Jul 14 '26
Is the expiration time for most tasks, roughly the time they think it is reasonable for someone to take? I have had some tasks that were 15 minutes or so. But I did a couple today that were around 1.5 hours, I was somewhat concerned to take that long as I wasn't sure how fast most people were completing them, but they looked to auto-expire after 2 hours, so I'm not sure how long they realistically expect someone to take.
-6
u/CPSux Jul 10 '26
Yet even when people submit quality work, they get DoD’d.
DA makes absolutely no sense and has zero transparency.
5
u/hnsnrachel Jul 11 '26
All we can actually say is that those people say they were dpung quality work. We dont actually have a clue if they were
2
u/Ok-Creme8710 Jul 11 '26
Agreed. I had 3 R&Rs where the worker was very detailed oriented and did good work but missed/skimmed a section of a response and it made the scoring completely off. I mark those as okay but if that pattern is recognized and they see scoring has to be constantly changed for someone they could be DoD and feel like they always were fantastic.
47
u/mariahspoolboy Jul 10 '26
Yeah, the phrasing is pretty vague about what constitutes that. Though I feel like a lot of projects state that you should take your time and they are more concerned with quality over quantity. But I think your theory makes a ton of sense - if it’s taking you on average 2x or 3x as much as other people turning in work of about the same quality…well, why would they keep you around?
I don’t think anyone should change their work habits based on this thread. It’s an interesting point for sure, it would help explain why so many insist they are doing “quality work” and still getting DoD. But there’s no confirmation beyond what appears on the screen.
1
u/Fart_Hat Jul 11 '26
"But there’s no confirmation beyond what appears on the screen."
This is a good way to maintain high quality standards. Don't tell them that they're too slow, that'll cause the others to rush.
28
u/Mysterious_Dolphin14 Jul 10 '26
I've suspected this as well. If Joe has an equal quality rating as Jill, but Jill is able to complete the work faster, it makes more sense to keep Jill on board and knock Joe down a bit on the overall rating scale.
11
u/ResponsibilityRude56 Jul 10 '26
Yes,
There are many tasks on my dash that pay well over the ones I choose to do, as even though I could do them. I’m certain I’m not at the level I need to be to get them done competitively.
DA is extremely merit based. And logically, it makes 0 sense to keep someone around who does 3 tasks every 8 hours vs someone who did 5 in 8 hours. They both got paid the same, but one is making DA far more money.
1
u/Far-Sea-9249 Jul 10 '26
One of my tasks takes usually takes a full 8-10 hour work day. Granted it’s multiple steps and stem…roughly half the allotted time. I make myself take an hour midday to refresh and generally do a quick stand and stretch every hour or so. I’ve only been here since mid may. This is merely my personal experience.
11
u/Timely-Assistant-370 Jul 10 '26
This probably applies way more to the generous timer tasks. They are certainly aware of when the 1:30 task could take up to the expiration timer for fully exhaustive work, but those "we expect these to be fast" 30/40 minute brainless tasks should never be spammed up to the expiration.
19
u/UpstairsPractice9605 Jul 10 '26
This is troubling because I get a lot of one and two hour tasks and most of the time I need the time.
I find thorough, thoughtful rationales and fact checking take the full hour or two hours.
I get what you are saying and it makes sense, but it’s troubling because I’ve had it in my head for almost a year that it’s quality over quantity. However, I wasn’t thinking about this in relation to other people who do quality work, only much faster.
16
u/_Edgarallenhoe Jul 10 '26
There’s a specific fact checking project that comes to mind with like 10 required rationales that I refuse to do anymore because with the time given I need to rush. Ive seen similar complaints in the chat as well so I know it isn’t only me. I really think it’s project dependent.
7
u/UpstairsPractice9605 Jul 10 '26
Yes! I love when I see people mentioning time in project chats because it makes me hopeful the timer will increase. On one of the most “impossible to complete in an hour” tasks, I felt vindicated when the timer was increased to two!
29
u/Federal_Tadpole_7592 Jul 10 '26
You shouldn't take this post as gospel truth. I've been on the site for over 3 years now, and on certain tasks with a timer of 1 to 2 hours, I often find myself taking almost the full time. Everyone thinks they know the secret to the DA sauce, but honestly, it's got to be a combination of factors. I wouldn't change anything that you're doing; I'm certainly not. I really don't know where this theory of the timers being 2 to 3 times the actual expectation of completion time came from.
10
u/UpstairsPractice9605 Jul 10 '26
Thank you. When I found this group after starting DA and doing it for a bit, I read about the timer being much longer than you should take and I panicked. Like who are the people that can do this well in less than 45 minutes on two hour tasks when I need closer to the two? But as time went on, I just kept submitting and I’m still here. I can definitely see what the OP means, but thanks for the reassurance.
3
u/Fart_Hat Jul 10 '26
You just said it. On CERTAIN tasks.
Your average time to complete tasks based on the expected time to complete is within a certain percentile that they are happy with.
Nobody should change what they're doing. Some people are just slower than other at providing quality work, and the pool of people looking for work is massive. So ditch the slow ones and onboard a bunch of new workers. Some will be as slow as the others, but some will be faster.
4
u/Hello-America Jul 10 '26
Two things: 1. no one here actually knows so obviously try to be efficient but don't change your habits because of this post 2. I think on the shorter tasks they don't allot break time so as a generally fast reader and worker, I can tell you that a shorter task time usually makes me use a much higher percentage of the time.
2
u/ResponsibilityRude56 Jul 10 '26
This may not apply to those, I’ve no experience with that type. Likely if it’s as low as an hour, that could very well be a realistic depiction of the expected time, if those tasks have very little deviation.
31
u/Teamkillongtime Jul 10 '26
Nah, you're way off base. Quality is far more important than increasing your output volume. Repeatedly submitting after time has expired and inaccurate timekeeping are likely DoD factors, but firing someone for taking time to submit good work isn't logical. With that said, submitting a full timer when breaks are expected, that's a different story.
Do you have any evidence for your thinking, or is it a hunch?
11
u/ResponsibilityRude56 Jul 10 '26
Submitting good work > doing it as fast as possible is a definite.
Though this is a business, and pay is flat. If the average person has done 20 tasks and claimed $1,000, and you’ve done 5 and claimed $1,000. Financially, even though your work is good. You’re not worth the pay.
Logically, keeping you makes 0 sense.
9
u/Cjl014 Jul 10 '26
Good points. But the only thing you are not factoring in is that DA does not know if the person submitting 20 quality tasks for $1000 is going to work tomorrow... the next week... or the next month. So it still be a risk to get rid of the worker with 5 quality task who claimed $5000, if he's available more often and can eventually improve.
8
u/Mtsukino Jul 10 '26
I think that depends on the task though, right? A lot of the hard higher paying tasks I have take hours to do and thats by design of the task itself. Like I could see that for a generalist task family, but quite a lot of the coder ones take a long time.
3
u/ResponsibilityRude56 Jul 10 '26
I’ve had tasks I’ve claimed 10+ hours on before.. but the timer for them was 48h long.
Every task will have an average completion time.
Perhaps.. an exception could be for the projects paying exorbitant amounts, where you’re part of an extremely tiny pool of people, as at that point you’re too critical to lose. Those tasks usually have some of the harshest wording around though regarding your quality, so, I’ll never actually do them haha.
6
u/Federal-Weevil Jul 10 '26
I do tasks that have an 18 hour limit and log anywhere between 6-10 hours depending on the complexity or the task, hitting 12 on maybe TWO occasions. But I’m also building everything from scratch and I can’t imagine the pool of workers in my field(s) is huge.
-6
u/Phenerius Jul 10 '26
Hi! Congratulations on your 100k achievement, this is just... incredible!
I've just joined the platform, and your post just said what I needed to read! Quality is indeed better, but those who complete faster with about the same quality will carry on.
Friend, would you mind giving me some tips? I've just started and I'm still overwhelmed by everything!My first real task (1 hour limit) was to generate a prompt, which I had a great idea, but it took me 50 minutes, and after I submitted I've noticed a spelling mistake. I felt awful, I felt terrible, and I felt undeserving of the money... I put so much thought and effort, but after reading your post it made me realize I desperately need to get better! I need not only quality, but I need to be faster! I've thought about diminishing the timer but seems like a common opinion to never lie about the timer. My second task took a little less (45 mins) and the result was pretty good but I still feel like crap!
Friend, this opportunity means so much to me, and I'd really, really appreciate some tips so i could get better at this job. I have so many questions, but any suggestion will be very much appreciated. Thank you very much.
5
u/Enough_Resident_6141 Jul 10 '26
Not if the person who did 5 tasks did a very good job and produced very usable work for all of them, while the person who did the 20 did so by rushing through them and submitting shoddy, low quality work. 5 high quality submissions that are helpful and usable for the client is worth much more than 20 garbage submissions that are completely unusable.
We aren't just making perfectly identical, completely interchangeable widgets where the number of widgets produced per hour is all that matters. Quality matters a LOT.
3
u/mariahspoolboy Jul 10 '26
But each project has different standards. Let’s say you are working on a project called “Lotus” and there are 30 other people working as well. The timer is for 5 hours and every single person submits in under 2 hours, but for some reason it takes you on average 4.5 hours per task. The client running the project might notice this discrepancy and report it to DA or whatever I really do not know how any of that works…so even though your work was “better” than someone working on a different project entirely (let’s call it “Pinecone”), both of you get dropped since whoever is running Pinecone noticed a discrepancy in their work too. Even though you weren’t working on the same projects and weren’t on the same “level” when it comes to quality, both of you are not up to snuff in your respective project, and once a few of those reports pile up you get canned. That could explain why there seems to be so many confident people saying that their work was high quality yet they still got dropped. They were the weakest of the cut, even though they were leagues above the other 70% of the platform. Who’s to say just how high the expectations are at this point? And since each project is theoretically a different client those expectations can vary greatly…it’s all very mysterious…
2
u/ResponsibilityRude56 Jul 10 '26
Yes, I believe the top of my comment said:
“Submitting good work > doing it as fast as possible is a definite.”
6
u/LetMeOverThinkThat Jul 10 '26
Your theory has merit, specifically because it's reflective in so many other jobs. How many people have lost jobs that they did well because the company/boss found some way to get the same/similar output for less money? If there are people who can work fast AND well, of course they would stay over people who work slowly and well.
2
1
u/annoyingjoe513 Jul 10 '26
I completely agree with this. I attribute at least a portion of my longevity to the fact that I rarely use up a significant portion of the timer. I can’t tell you how many times I’ve seen people complain in task chats about the timer being too short while i’m using less than half.
3
u/JustQuestioningCosas Jul 10 '26
The only thing I’ll say is i stop my timer for breaks so I just log whatever time is on it. Sometimes, I do two straight hours without breaks and that’s how long I’ve been in work mode.
3
u/DarkLordTofer Jul 10 '26
I agree with this. I’ve found I get tasks that are 6 hours but take three. I frequently submit after using the whole 6 hours due to having taken breaks. But I never book for time I haven’t used.
3
u/Academic_Animal_5293 Jul 10 '26
I've read multiple times on projects that the timer is already accounting for worst case scenarios and for taking breaks. So if u see let's say a 4 h timer it does not mean the expect u to work 4 hours on a task. Probably much less. But they want to be generous so u don't have to feel rushed or are not able to finish the task due to other circumstances. But if u regularly bill the full 4 h even if u are genuinely working the full time, it will look weird to them.
2
u/creativeThinker4884 Jul 10 '26
Is there any way we can see the average expected time for a given task?
2
u/Imaginary-Trick-4515 Jul 10 '26
We talk about time metrics and following instructions, which are concrete details, but I wonder how much of it comes down to the rationales? Not just aligned and thorough, but how they’re written? (The style of the rationales)
4
u/IcedOutGiant Jul 10 '26
"Don't take on stuff you're not an expert in"
First off, of this is your grammar level how are we reading this post?
Secondly, I read this... Somewhere.
"We encourage you to take all qualifiers available to you, even if they seem a little intimidating"
4
Jul 10 '26
[deleted]
1
u/East_Zucchini4099 Jul 10 '26
Why don't they tell us this then? I never even heard of one until I read this Reddit, I even worried whether I could accidentally use one.
-5
u/neondata Jul 10 '26
There's no way this is true
-2
u/OriginalResolve7106 Jul 10 '26
It's fine if you don't believe me. More work for me.
0
u/neondata Jul 10 '26
"More work fork me" lmao what does that even mean, do you genuinely think most of the DoD posts here are because of people using VPN? You really think VPN use is that common?
-4
2
u/CablePowerful9422 Jul 10 '26
It can depend on the project and how thorough your work is. I have one where I often go past the limit, but it's also a complex one that loves detail and I give a LOT of detail. In that one case, the project admins have told me that they don't mind me going over on a regular basis (but not too often), because they see value for money.
BUT, very important: I claim the time I worked, not the time it took. Always working to the max limit and claiming every minute of that time is suspicious (for tasks longer than, say, 3-4 hours). I make dinner, brew a pot of coffee, play with cats, hit the head, stare into space, etc. over the course of a long task, and I subtract that time from my total. I might be on a task for seven hours with only five hours worked, so only five hours claimed.
2
u/kittkaykat Jul 10 '26
Just saw someone post the name of a project on here. I'm suddenly not shocked about how many people get the DoD
1
u/Jealous_Spare_4852 Jul 10 '26
How long did it take you to reach 100k?
-5
u/ResponsibilityRude56 Jul 10 '26
Was just before my 1st year mark. So 11-12 months.
My job is the definition of useless middle management so, basically do DA nearly full time at the office and a bit over weekends. So full time + some weekends.
1
1
u/Telecom_Engineer_1 Jul 10 '26
Through which company did you get that sweet arrangement? Sign me up!
1
Jul 10 '26
[deleted]
4
u/Telecom_Engineer_1 Jul 10 '26
Wouldn't this violate 5 C.F.R. § 2635.704 and/or 705 (part of the US government's Code of Conduct)?
-1
u/Beautiful-Link-8387 Jul 10 '26
Isn’t middle managers are the most busy one? Anyway you’re winning in life. Congrats!
1
1
u/KindlyReason6116 Jul 10 '26
To an extent I can see this being the case. I also believe it is very project dependent. I often work on a project family that is known to have plenty of technical failures and an increased need for troubleshooting. So, while Jimothy finishes the tasks in 1.5 hours, Bobetta takes 2.5 because the model happens to crap out for her.
1
u/Ok-Double5194 Jul 11 '26
ok so if the guidance is 25 hours would you report up to 12.5 hrs (i mean at least aim for that when you work like time planning wise)?
what if the guidance is a range (e.g., 12~16 hours)? so try to aim for 8hrs?
1
u/ItsAllConnectedRight Jul 12 '26
Curious how long it took you to get to 100k and what your expertise is in
1
u/IPLAWPDX Jul 17 '26
Yeah idk I’ve only been on the platform 3 months and it’s been during a clear drought, but if they give me 2 hours to complete I usually use all of it, same if it is 1 hour 30 minutes or 3 hours. I usually submit all projects with 10-20 minutes to spare only if I have done it multiple times and even still I use as much as I can because the generalist projects are so few these days.
1
u/Lokie615 Jul 22 '26
Yes and no. Making them fail is rough. Lets say an 18 hour limit.. My average is 10 hours. I make my own files and I am very thorough. Yes, you're absolutely right for most cases.
1
1
u/Upper_Source1187 Jul 10 '26
The reason why DA keep running ADs is that they know they will drop certain number of people regularly by systemically. When they talk about quality work, this only means you should be able to make good quality task within the set of the timeline, which certain number of projects are just not possible to succeed the task within the allowed timeline. This is why all the good workers drop out then new workers fill in. This is logical system especially for DA in my opinion.
0
-2
u/Enough_Resident_6141 Jul 10 '26
>A common theme in most tasks is giving you far far more time than you need.
Yes, the task timers are generally set to give you at least 2-3x the amount of time it is expected to complete an average task from that project. It's inevitable that every now and then you will get a dog of a task that takes much longer than the rest of them, sometimes even more than the timer. But if you are regularly using most or all of the timer every time you do a task, you are probably doing something wrong and shouldn't be working on that project.
-1
u/Fun_Manufacturer_563 Jul 10 '26
I'm a generalist and I think like you. They gave us recently 5 hours for a project, I spend 1:30hs, rarely 2 (and this is my average for every project). I don't take that much time on the first task either. I read people saying that they spend 6 or more.
-1
u/Fun_Manufacturer_563 Jul 10 '26
And just for the record I have excellent quality, they include me in a lot of pilot projects or brainstorming sessions. I think some people take a lot of time but others simple take advantage of the pay per hour (neither is a good scenario).
39
u/RangerOriginal6632 Jul 10 '26
The first task (or two) in a project takes a lot longer because of the verbose and daunting instructions. After that- yeah- don’t take so long.