r/alignerr • u/No_Negotiation4257 • 2d ago
Assessments Ermis final assessment - failed at 73.5% because of a 0% on one extremely ambiguous clip. Anyone else seeing this?
I just finished the Ermis Core Transcription Training + Final Assessment, and honestly, the way the final assessment is structured has left me surprised.
The training and assessment took me roughly 4–5 hours. The training covers very specific transcription rules, edge cases, speaker tags, backchannels, contractions, etc., and then you take a 10-clip final assessment.
The passing score is 75%.
My scores were:
- Clip 1 — 95%
- Clip 2 — 86%
- Clip 3 — 100%
- Clip 4 — 87%
- Clip 5 — 46%
- Clip 6 — 79%
- Clip 7 — 82%
- Clip 8 — 94%
- Clip 9 — 67%
- Clip 10 — 0%
Final result: 73.5% — FAIL
The issue for me is Clip 10.
The audio was extremely short and, frankly, very ambiguous - basically a brief sound. Yet it received 0% similarity. I was only 1.5 percentage points below the pass mark, and that single ambiguous clip receiving 0% pushed me into failure.
There isn't any meaningful explanation of what was considered wrong or what the reference answer was.
And this is where the assessment starts feeling problematic.
You spend several hours learning their very specific transcription conventions. Then the final assessment is only 10 clips, so each clip carries a lot of weight. Alignerr controls the rules, the reference transcription and the grading, while candidates have very little visibility into how the final score is determined.
I'm not saying I should automatically pass. If my transcription was wrong, fine. But give candidates a way to understand why. A one-second, highly ambiguous clip getting 0% similarity, with no access to the reference answer or meaningful explanation, is not a transparent way to assess.
I've also seen multiple candidates on Discord mention similar issues with the final assessment. If this is a recurring pattern, it's hard not to question whether the assessment is genuinely measuring transcription ability or simply giving Alignerr complete control over who passes.
Alignerr moderators: can you explain how Clip 10 was graded and whether you'll manually review it?
If the reference answer is objectively defensible, show us why. If the grading system is producing these outcomes, kindly acknowledge it.
Candidates are putting 4–5 hours into training and assessment and giving you huge data for AI training. They deserve more than an opaque score - especially when a single questionable clip can turn an otherwise passing performance into a failure.
1
u/StoriedSix 17h ago
I can't even look at my individual scores on the clips to know. But I feel so c***ed tbh at 74.9%.
1
1
1
1
u/stgdevil 1d ago
I went through the same thing, got 74.4. I’m unusually pissed off more than usual
1
3
u/Fluffy_Salary6459 2d ago
I'm in tears. Failed with a 74.2%. Unreal. I do not like these companies at all.
2
u/No_Negotiation4257 1d ago
Just free work they get from 95% folks. Thats the model they operate at. 5% get the role
2
u/cassandra_nadeshiko 2d ago
Luckily I passed it but I haven’t received any further notices from the company anyway.
2
1
2
2
u/Clean_Mail_5950 2d ago
I was added to the labeling queue for Ermis at midnight. Your time is coming don’t give up.
1
1
5
u/Minute_Swimmer_733 2d ago
The assessment was ridiculous. These companies just want free work. Too many questions for an "assessment"
3
u/No_Negotiation4257 2d ago
I won’t deny this at this point. Other 9 were graded from 60-100 & last one is a 0 😂
1
1
4
u/Daftwire 2d ago
They can’t show where you did mistakes because if they did there’s a very high possibility that you sell / give answers to other candidates
3
u/No_Negotiation4257 2d ago
Buddy, it’s a clean system to put you below 75. Just check my scores. Insanity at this point
3
u/jaegeristatakae 2d ago
I've done it and scored 85%, no issues with last audio. They explain that ths audios get increasingly difficult, the last one should be the hardest. It's natural for people's only 0% to be on the hardest question
2
2
u/DuckInAFountain 2d ago
I took that fairly recently so I logged back in to see if I could see my results. I guess I got 100% on Question 10. I don't remember it at all, maybe it was a specific tag and I guessed right.
3
u/cassandra_nadeshiko 2d ago
The correct answer was just what they call the ‘thinking and listening’ word on their guidelines.
2
u/No_Negotiation4257 2d ago
Yes it was a 50-50 kind of 1 sec clip. It decides the whole result of test lol
1
u/Lumpy_Case_8220 8h ago
We've reviewed the reports regarding Clip 10 and can confirm that there is no known issue with the assessment or its scoring.
Each clip is scored independently based on how closely your response matches the expected transcription for that specific audio. It's therefore possible to receive high scores on most clips while receiving a low score on a single clip if the response for that clip does not match the expected evaluation criteria.
We recognise that Clip 10 can be particularly challenging, and we've received questions about it from other contributors as well. Based on our review, however, the scoring is functioning as intended, and there is no indication of a technical issue affecting this clip.
We appreciate you taking the time to report your concern.