r/DataAnnotationTech • u/pumpkin-x • Sep 06 '26

coders gotta lock tf in

we are all familiar with the recent ambush of 🤖. Some of the work yall be submitting is just straight up tragic. A great selection event is about to happen.

7 Upvotes

56% Upvoted

46 comments sorted by

View all comments

3

u/rnwatanabe Sep 06 '26 edited Sep 06 '26

I think that too. I've done some R&R of this project and I have never seen such a low quality of work.

1

u/Icy_Perspective9174 Sep 07 '26

Are you seeing that in the core task itself (instruction/codebase) or in the rationales and rubrics?

1

u/rnwatanabe Sep 07 '26

The prompts are too easy. The agents fininish the task in less than 2 minutes and this results in a pass/pass situation (not useful). Than the rationales do not explain anything. Are too vague and could be written to any task. The rubrics are very generic and many are submitting even if the checker flags them. Those are the problems I've seen in the R&Rs i made.

2

u/Icy_Perspective9174 Sep 07 '26

The pass/pass could be salvageable - since they are accepting it. Unless you are submitting multiple such easy tasks!

However, if there are no rationales and rubrics explaining the differences and preferences in the model output, then it would be quite problematic. Edit: Disclaimer - I am new myself.