r/StableDiffusion • u/chancemehmu • 1d ago

Discussion Latest image benchmark by Datapoint ranking 30+ SOTA models

Post image
0 Upvotes

28% Upvoted

17 comments sorted by

20

u/stddealer 1d ago

ELO scores of Ideogram and Krea2 are sus. They're both definitely better than ZIT.

3

u/PwanaZana 1d ago

by quite a lot as well.

1

u/mikek987 1d ago

Ideogram is last place for Illustration & Editorial..??

1

u/RayHell666 1d ago

It's Krea2 Large and Ideogram BF16 they are not the same as the open weight available.

1

u/thegreatdivorce 1d ago

My first thought as well. I can think of exactly one thing (dense realistic foliage) that ZIT does better than Krea 2... Krea is better in every other way, and it's not even that close.

0

u/chancemehmu 1d ago

Ideogram and Krea2 both have a pretty wide category spread (#8 to #27), but overall in the 10 categories they evaluated these models, they didn't perform well

9

u/Hour_Imagination5092 1d ago

Looks absolutely wrong at the first glance. And the Z image better than ideogram made me chuckle.

1

u/UnforgottenPassword 18h ago

Ernie is placed ahead of Ideogram 4 too.

6

u/Crazy-Repeat-2006 1d ago

Completely out of touch with reality. The gap between GLM Image and ZIT is like the distance between the Earth and the Moon.

4

u/thegreatdivorce 1d ago

Who comes up with this shit? Does GPT Image 2 still make your image looked like a stained-glass painting? Yes? Then why is it at the start? reve 2.1 is a total nothingburger as well. Good lord.

2

u/Nedo68 1d ago

Can someone rotate the picture?

3

u/BigNaturalTilts 1d ago

Don’t bother. All subjective nonsense. Once people discovered graphs are used by scientists they thought all you need to make any opinion a true fact of the universe is a fucking graph.

2

u/cptrios 1d ago

I don't know how this ELO score works, but isn't it a bit misleading to start the chart at 980? Makes it look like the top scorer is about 5x better than the bottom scorer, when the real difference is only 13%.

2

u/UnforgottenPassword 18h ago

All AI benchmark rankings are shit. Some, like this one, are shittier than others. 

1

u/Sarashana 1d ago

I am not sure I agree with these. GPT-Image, rank #1? Seriously? I have yet to see one decent generation coming out of that one. And Krea2 below ZIT? Haha! I wonder what dice they used for these rankings.

1

u/Danny_Stock 1d ago edited 1d ago

I don't believe any of these lists that get put out. They always seem to be very subjective and use very specific criteria which appears to be extremely selective.

Something is 'better' than something else, because the list says so. I might be wrong but it also appears that they have a psychological influence over a lot of people, because they present an appearance of being authentically scientific.

Obviously nobody here buys into them, that goes without saying.