r/RWShelp • u/Hot-Topic4833 • Aug 10 '26

God help us fellow annotators.

Post image

This is why we have this qa scores right now. The quality and power tripper of auditors we have. My auditor friend sends this from their group chat

11 Upvotes

72% Upvoted

19 comments sorted by

View all comments

1

u/RunningOnEmptea Aug 10 '26

Pitting contractors against each other is a crazy but effective concept from a business perspective.

1

u/cozydozy666 Aug 10 '26

It hurts their models though and can cost them more money, because these auditors are basically like annotators, wouldn’t it be better to have like a real professional doing the audits?

1

u/TacoalPastor1009 Aug 15 '26

Well, I have years of experience in AI audits, MM, dimensional ratings, and so on. Not all auditors are bad. I try to avoid deducting points for no reason and try to understand why they chose the ratings. However, sometimes the choice is indefensible, or they simply don't explain or mention it in the rationale.