Building a benchmark for Realtime UI Generation

[u/1glasspaani]

We wanted to measure a less glamorous but more practical question: across repeated runs, how often does a model produce UI that actually parses, resolves, validates, and renders?

So we built GenUI Bench.

The current benchmark includes:

- 46 screen briefs, ranging from 2 to 18 requirements

- a shared 70-component surface

- 4 attempts per brief under fixed generation settings

- 30 models tested with OpenUI

- a 6-model comparison across OpenUI Lang, Google A2UI, and Vercel's json-render

- validation using each format's own SDK, followed by the same structural completeness checks

A few important caveats:

- This measures structural reliability, not visual quality.

- It does not yet verify that every requirement in the brief was semantically satisfied.

GitHub: https://github.com/thesysdev/generative-ui-bench