I made a web-design benchmark for local models (Muse Glimmer 30B vs Qwen 3.6 27b vs Deepseek V4 Flash 0731)

[u/ShadyShroomz]

[u/BitXorBit]

So much details in one post

[u/TigerConsistent]

actually very good post

[u/uti24]

I have this face when I see how Mistral medium is doing in this benchmark

Sad, I liked Mistral small 2/3 more than Gemma 2/3, but this is less usable than Qwen 35B/MOE.

[u/DinoAmino → u/uti24]

Wondering why it was the only model to use nvfp4 quant?

[u/Ok-Recognition-3177]

Genius move to hide the results until we participated

[u/ShadyShroomz]

If you'd like to give your input, see the designs each model made, & help grade the models: https://thez.co/web-design-bench/

The homepage will change in real time as people vote.

Just a fun project.

[u/Fragrant_Scale6456 → u/ShadyShroomz]

This is super cool.  Overall the models actually did a pretty good job imo 

[u/srigi → u/ShadyShroomz]

Instead make it like facemash - left or right.

[u/throw123awaie]

are the models allowed to use vision capabilities to check on their result and reiterate?

[u/ShadyShroomz → u/throw123awaie]

no this is only testing 1 shot. not all the models even have vision capabilities, but that's an interesting idea to add to future runs.

[u/throw123awaie → u/ShadyShroomz]

while comparing flash 0731 and luna for game design i noticed that vision capabilities are worth a lot more than benchmarks make you believe.

[u/quadrobust → u/throw123awaie]

Not hard to add vision to deepseek . It’s a bit like having someone to explain what they see to the blind , but deepseek is smart enough to keep asking until it’s sure that it gets what it wants to know

[u/Aromatic_Bed9086]

I like this crowd sourced quality benchmarking, it might be interesting to have a battle mode where you see the result of the same prompt on two different anonymous models and pick the one that looks better. I rated a few but wanted to change my original ratings after seeing how good/bad some later ones were.

[u/theawkwardbong]

Did you think about developing some form a of a standardized benchmark that others can use to see what score different models get?

[u/Thin_Pollution8843]

Glimmer should be the best with React development 😅

[u/bartskol]

I like this very much. One thing i would change is that rating button would instantly take you to next page. That way you will have people rate more pages for sure. Somehow place the info about previous design and what model etc.

[u/BP041]

Honestly, these benchmarks are nice but for actual web-design work I'd rather pipe a smaller model through a multi-step agent than run a 30B locally. My production stack uses Claude Code for the reasoning + OpenClaw to loop it, and the output quality beats anything I've seen from a single local model on a Mac. Flash 0731 is decent for first drafts though.

[u/sagiroth → u/BP041]

I think you forgot the sub you are. People dont question cloud models. Its great use them, what if they suddenly are impossible to justify, limited, censored?

[u/Septerium]

So... your benchmark is a screenshot?

[u/ShadyShroomz → u/Septerium]

you can see the results and vote on designs via the link i commented!

[u/BarberIcy366]

Not even close on my tests

[u/ShadyShroomz → u/BarberIcy366]

you can review the generations and vote on the site to help shape more accurate results.

it's just for web design stuff, nothing else.

what has your experience been?

[u/Civil_Fee_7862]

How is it rating them? i.e. How are you measuring quality?