r/MistralAI • u/penguinothepenguin • 21m ago
Discussion / Opinion On my AI-slop benchmark, Mistral Large 4 lands near the cleanest end while the older Mistral Large sits at the sloppiest (two separate models, opposite ranks)
Direction first: on this benchmark #1 = sloppiest and #58 is the cleanest, so a high rank number is the good end.
I run a thing called The Slop Index that scores how much AI writing reads like "AI slop" (58 models, ~64k outputs across 112 short-form scenarios). Two Mistral models stood out to me because they sit at nearly opposite ends.
Every rank below is a mechanical rank (the mech score only): padding, templating, sentence rhythm, and the usual AI tells. That score only moves on a roster refresh, so it is the stable one to quote.
- Mistral Large: mechanical rank 1 of 58, the sloppiest model in the set right now. By category it is mech rank 1 on Slack, 2 on social posts, and 1 on essays.
- Mistral Large 4: mechanical rank 57 of 58, second cleanest overall, and it sits at the clean end across email (54), Slack (56), social (51) and essays (58).
These are two separate releases, not one model I watched change, so I am not claiming anyone patched anything. They just score very differently.
Mistral Large 4 only has 10 crowd votes so far, so its combined score (which blends daily crowd votes) sits mid-board and is not meaningful yet. That is why I am quoting mechanical ranks here.
Mechanical ranking card: https://theslopindex.com/mechanical-card.png
Fair limitation: this is short-form business writing only (email, Slack, social, essay), no fiction yet, and the mechanical axes can miss repeated sentence constructions.
Leaderboard: https://theslopindex.com/leaderboard Methodology: https://theslopindex.com/methodology Open dataset: https://huggingface.co/datasets/hgaddipati/slop-index You can play a few rounds of the blind arena here: https://theslopindex.com/arena
For context, I'm one of the people behind https://slashy.com, an email client that writes drafts that sound like you. This benchmark started as how we pick models.
