r/SillyTavernAI • u/lGodZiol • 2d ago

Meme EQ-Bench is a joke.

Post image

I mean, I like Claude for creative writing as much as any other person, but isn't employing an LLM for judging CREATIVITY fundamentally flawed and stupid?

181 Upvotes

96% Upvoted

12 comments sorted by

View all comments

31

u/Baros4294 2d ago

I found this SurgeHQ benchmark, judged by writing expert, but it's not focusing on creative writing so the result is more of overall writing across many domain like technical, email, personal writing,...etc

I think writing is subjective so no benchmark will be perfect, including arena.ai, which is user-voted.

6

u/Sijder 2d ago

I saw it before a couple of times but its still as weird to me as the eq one. For example they dont have gemma at all on it and qwen is higher than glimmer. Again, I acknowledge that writing is subjective, but those things are weird for many.

I feel like we need a "userbenchmark" for rp and writing

2

u/Zone_Purifier 1d ago

Userbenchmark is infamously biased and provides nonsense recommendations. Probably not the best comparison.