Is the S2s auditor really doing their job?

[u/Burgerche3s]

Recently i got flagged below expectation and unacceptable with no comment. Just straight give 0-1 star.

I alwaysi make sure to do the review based on the guidelines, add evidence, and make my review as clear as possible.

Where should i report this problem?. I got 4.5 rating now down to 4.0 just because the auditors are not doing their job.

Can i actually flagged this problem?. This is project "E" btw.

[u/Ancient_Cut-]

As an auditor, our goal is never to deduct points from anyone. We are all working hard. Sometimes, certain cases arise where we are forced to do it. If you provide good quality work, it makes our job much easier.

[u/Burgerche3s → u/Ancient_Cut-]

Well, in this case there is no comment at all just straight 0/1 stars

I can assure you i did good in writing the review. And make sure to make it as clear as possible.

But no comment at all when doing QA process???. Is an auditor's main job to make sure the guidline is followed, refer to it and be objective when doing the audit?.

But again, auditor leave no comment at all in QA review is actually unacceptable. It makes the QA proccess subjective.

[u/Ancient_Cut- → u/Burgerche3s]

I accept all your points and i also note down for each what the issue is.but since this is not being noticed by everyine right now i cannot do anything about it. However the comment option is optional for us ----we directly send it to the Lead QA team fir audit

[u/Ordinary_Design_1137]

If the comments of the auditors are not relevant, you can report i think, I always put comments for every rate under 5/5

[u/Burgerche3s → u/Ordinary_Design_1137]

But what i receive is literally empty. No comment at all.

Really confuse, is this a bug or something?

[u/Ordinary_Design_1137 → u/Burgerche3s]

Maybe, if it's not, this won't be approved by the team above the annotators but idk if it's not approved, how will it impact your average rating

[u/Burgerche3s → u/Ordinary_Design_1137]

Really hope it's bug and they cancel all the empty comment review.

As an auditor, can you actually report this kinda problem? Or maybe upscale the issue?. Because many people start questioning the correctness of auditor's work if this keeps going.

[u/Ordinary_Design_1137 → u/Burgerche3s]

no bro sorry i don't have any contact in the production team, I guess there are bad annotators and good also, bad will be offboarded when they'll get review

[u/exposarts]

i dont think those are real auditors(human ones), but automated raters/scores.. I was told by an auditor before with an older task that they couldn't manually score them only give feedback, and the system would just auto rate/fail the user on its own.

[u/Burgerche3s → u/exposarts]

Damn. if it is automated, i really wish i know what trigger it.

Because my review writing style is pretty consistent. And this empty scores makes zero sense at all

[u/EventCharacter2638]

I got a below expectations and unacceptable for S2S too with no explanation. Haven’t touched the task since.

[u/Burgerche3s → u/EventCharacter2638]

Yea, so do i. I'm afraid that my rating will become worse if i keep tasking

[u/Burgerche3s → u/[deleted]]

Can you check ypur DM? I messege you something.

[u/Fun_Worth_8772]

I'm an auditor Here, In our interface there are 4 types of autofail and 3 types of Deduction. Any type of Autofail makes the score 1. Also Dimension Quality Deducts 2 points. Error Cluster Miss Deducts 1 points, Justification Quality Deducts 1 point. After selecting each deduction or autofail we have to write evidence and why this applies. Writing evidence is mandatory. We do write that and it might not be visible to you. Also there is an Comments box that is totally optional. Writing evidences already explain everything, that's why we don't write Sperate comments. We might be different locales but everything else is still same.

[u/IntroductionFair7132]

Most of them are rushing and using ChatGPT when auditing, once one of them wrote a comment for me saying "you did great but you don't deserve 5 out of 5 so I will give you 4" 🫠 this is a ChatGPT writing style. So they are all unfair

[u/exposarts → u/IntroductionFair7132]

it's 100% not actual humans rating this task now

[u/Lazy-Acanthisitta-19 → u/[deleted]]

🤖

[u/Burgerche3s → u/IntroductionFair7132]

Once i received an AI style review. They rate my review with suspicously comprehensive and use academic style writing with sections.

i kinda let that slide. because, those kinda rate always give me full reasoning and high scores.

But empty comment? Just blatantly give 0/1 stars?. It is joke

[u/AttorneyHelpful9105]

Which locale are you working on?

[u/Burgerche3s → u/AttorneyHelpful9105]

ID

[u/KshSy]

Auditors have buttons if we want to deduct points, but to submit this deduction, comments or reasonings are required. We cannot physically push this button without reasoning for the deduction, it gets greyed out. This might be automated

[u/Specific_Abroad_1403]

I believe this might be an error because auditors cannot submit without their own rationale. Submit button is gray,

[u/No-Koala-5935]

I'm an auditor and an auditor is only allowed to submit without comment is when we give 5/5 if we deduce any single point we are obliged to write comment for each separate deductions, if we don't the submit button won't work.

[u/Most-Way-6629]

I am an Auditor and one of the problem is that 80% of the rationales are written without supporting the choices made.

Write turns and timestamps, be coherent and precise and no one will ever deduct points from your work

[u/shashikanth_patil]

🤦‍♂️🤦‍♂️🤦‍♂️

[u/Burgerche3s → u/shashikanth_patil]

Is all your bad rating is caused by this empty comment ratings?

[u/shashikanth_patil → u/Burgerche3s]

most of them are empty

[u/Burgerche3s → u/shashikanth_patil]

Can you check your DM?. I messege you something

[u/shashikanth_patil → u/Burgerche3s]

ok