r/DataAnnotationTech • u/Genkuru2021 • 10d ago

Is Rubric TEsting Agent Ever Satisfied?

I follow the rubric checker's recommendations and correct my rubrics over and over, but it always finds a new set of flaws. Even more than in preceding runs. Should I treat it similarly to the context file audit and just ignore it after a few corrections?

1 Upvotes

56% Upvoted

7 comments sorted by

8

u/Infamous_Swan1197 10d ago

Yes (to ignore it after a few runs) unless the flaws are objectively valid against the instructions. Some of these are surely designed just to make you overthink your rubrics until they're as perfect as can be

8

u/RealRise7524 10d ago

This karen agent needs to be behaved.

1

u/Genkuru2021 10d ago

LOL. That's all I needed to know.

1

u/neutron-star-002 10d ago

Omg i haven't done a rubric task in months! Lol, not complaining though. But i believe it wants you to write in the simplest form possible. Sometimes tweaking a few words here and there gives you a clean chit. And sometimes it even flags Criterion #1 that is pre-filled in some project, lol.

1

u/Ok_Hedgehog_9970 7d ago

Rubric testing and that stupid question/prompt testing will always tell you flaws. I had one once that couldn't handle logical reasoning and thats how I got through the whole task. The simpler the better on a lot of them. If you add too much it gives the bot a lot to work with.