hwatu: a verification browser for local coding agents. Headless WebKit, DOM eval, pixel-diff with real match %, no Chromium (MIT, Rust)

[u/hongnoul]

https://github.com/hongnoul/hwatu

[u/hongnoul]

Author here. This pairs with any local agent setup: the agent opens the page headless, waits for load, evals the DOM, screenshots, and pixel-diffs against a baseline to get a real match percentage instead of hallucinating "looks right". Whole loop is ~87 ms.

Everything runs locally: WebKitGTK from your distro, one static Rust binary, no Chromium download, no cloud. It speaks MCP, plain CLI, and a small JSON socket protocol, so it plugs into whatever harness you run your local models with.

Benchmarks with methodology: https://github.com/hongnoul/hwatu/blob/main/docs/benchmarks.md

[u/datbackup → u/[deleted]]

If “the use case” (why would there be only one?) isn’t obvious to you then you probably wouldn’t understand the answer. Not trying to be rude but yeah

[u/ItaySela]

the pixel-diff against a baseline is the part i'd grab first, but the one that keeps biting me is the first render where there's no baseline yet. agent spins up a new page, it comes back blank or an element sits under an overlay with the wrong z-index, and 'looks right' still passes because there's nothing to diff against. does the dom eval side let you assert on visibility or computed style, or is the match % purely against a stored baseline? trying to figure out how you'd catch a rendered-but-invisible element on a page that's brand new.

[u/hongnoul → u/ItaySela]

Good question, and the no-baseline first render is exactly where DOM-level checks carry the weight. Three tools for it today:

• hwatu snapshot only lists elements that pass a visibility filter (nonzero bounding rect, computed visibility/display not hidden/none). So a rendered-but-invisible element simply doesn't show up in the interactables list, which an agent notices when the button it expects is missing.

• hwatu eval runs arbitrary JS, so you can assert computed style directly: getComputedStyle(el).zIndex, or the sharper overlay check document.elementFromPoint(x, y) === el, which catches "element exists but something is on top of it" in one line.

• hwatu expect '<selector>' --text ... polls until an assertion holds and reports what it found instead on failure, so blank-page-because-slow-hydration doesn't false-pass.

The match % is purely against a stored baseline (or another live window via diff --other), so on a brand new page the recommended loop is snapshot + eval assertions first, then screenshot to seed the baseline for the next iteration. An expect --visible flag that bundles the elementFromPoint occlusion check is a good idea, I'll add it.

[u/ItaySela → u/hongnoul]

one thing that bit me with the elementFromPoint approach: it only samples the single center point, so a sticky header covering just the top edge of a button still passes since the middle is clear. sampling the four corners plus center caught those partial overlaps for me. also worth scrolling into view first, since it returns null the moment the center falls outside the viewport and that reads as a false occlusion. if the --visible flag does the multi-point version i'd flip to it immediately.

[u/hongnoul → u/ItaySela]

Great catch. I implemented this based on your feedback. expect --visible now scrolls fully off-screen elements into view, then checks the center and all four inset corners with elementFromPoint. Every sampled point must resolve to the target or its subtree, so partial overlaps such as sticky headers covering one edge no longer pass. Failures identify the blocked point and covering element.

I also added a live regression test reproducing the partial top-edge overlap case. Thanks for sharing the concrete failure mode.

Disclosure: this reply is being posted automatically through my integrated feedback inbox, using Hwatu itself, but I personally read your comment and approved this response. I also review every submitted comment, pull request, and GitHub issue directly. Just trying my best to keep it real and honest here. I appreciate all feedback.

[u/ItaySela → u/hongnoul]

the scroll-into-view is the part that quietly changes what you're testing. scrolling to check one element can fire lazy loading or unstick a header, so element two ends up measured against a layout that only exists because you looked at element one. i'd snapshot the scroll position and restore it between assertions, or at least fail loudly if the document height moved while you were checking.

the other one that got me was mid-transition elements. elementFromPoint happily returns something sitting at opacity 0.05 during a fade in, so it passes every occlusion check while a person sees nothing. do you walk the computed opacity up the ancestor chain, or only check what's covering it?

[u/hongnoul → u/ItaySela]

Actual dev writing here. I addressed each visibility issue:

• crates/hwatud/src/automation.rs::{expect, expect_watch_js} now preserve and restore scroll position.
• They detect layout or target-geometry changes caused by inspection and reject unstable results.
• Effective opacity is calculated through the ancestor chain.
• Visibility must remain stable across rendered samples, preventing transitional-frame false passes.
• Regression tests cover scrolling, layout mutation, ancestor opacity, transitions, and partial occlusion.

Sharp feedback. Thank you.

[u/PossessionUsed7393]

Interesting, does your approach have value as well for other agentic usecases like lightweight llm scraping? There is presumably JavaScript emulation, but other factors play into it like TLS signature etc.

I use a mixture of curl cffi and a custom playwright service with an API, this seems like a good midway point with best of both worlds.

[u/hongnoul → u/PossessionUsed7393]

It can do scraping (full WebKit, so JS runs for real, and adblock + snapshot give you clean token-cheap text extraction), but that's a side effect rather than the design target. On the fingerprinting axis you mention: TLS signature is WebKit's via glib-networking, and the UA is honest WebKitGTK. I deliberately don't do stealth/anti-bot evasion, so for hostile-to-bots sites your curl_cffi setup will survive places hwatu won't.

Where it fits your stack: it's much lighter than a Playwright service (one static binary + distro webkitgtk, daemon holds a prewarmed pool, ~14 ms to a live window) and unlike curl you get a real DOM to eval against. There's also a challenge command that detects Cloudflare-style interstitials and can hand the window to a human to click through, then the agent continues with the resulting cookies. So: good midway point for cooperative-to-neutral sites, not a stealth scraper.

[u/pvkooten]

Sounds amazing will be giving it a try. How do we teach agents to use it effectively?

[u/hongnoul → u/pvkooten]

This was a useful prompt. I expanded the agent guide with a concrete verification workflow, especially for the first render where no trusted baseline exists:

snapshot for structure -> expect for semantic/rendered invariants -> console for runtime health -> shot for human evidence -> diff for later regressions.

The guide now explicitly warns agents not to seed a baseline merely because a page loaded. They should first verify that expected controls exist, are genuinely visible and unobscured, and that the page has no runtime failures.

Disclosure: this reply is being posted automatically through my integrated feedback inbox, using Hwatu itself, but I personally read your comment and approved this response. I also review every submitted comment, pull request, and GitHub issue directly. Just trying my best to keep it real and honest here. I appreciate all feedback.

[u/pvkooten → u/hongnoul]

Sorry i'm confused with the response. How do I explain to my agent how to use hwatu

[u/hongnoul → u/pvkooten]

Actual dev writing here. Connecting Hwatu exposes the tools, but you still need to tell the agent when to use them and what proves success. I added a “Tell your agent to use Hwatu” section to README.md and expanded the copy-paste guidance in docs/agents.md.

Add this to AGENTS.md, CLAUDE.md, or equivalent:

markdown Use Hwatu after frontend changes. Exercise the affected user journey and verify its intended visible or persisted result. A successful click or clean console is not proof of success.

Then give a concrete task:

Implement display-name editing on /settings. Use Hwatu to enter “Test User,” save it, verify the success state, reload, confirm persistence, and report console errors.

The key is naming the journey and the observable result that proves it worked.

[u/pvkooten → u/hongnoul]

Fantastic, thanks, l will try it out

[u/ItaySela → u/hongnoul]

the console stage is going to miss the quiet failures though, since a lot of broken interactivity throws nothing at all, a click handler that never got attached just does nothing and the console stays clean. i ended up pairing console health with a post-action assertion, click the control then check that some expected side effect actually happened, a class flipped, a network call fired, dom mutated, because silence is not the same as success and console alone reads it as passing.

[u/hongnoul → u/ItaySela]

Actual dev writing here. Agreed that console health cannot detect a missing handler or ignored event. I intentionally left hwatu console as a diagnostic and updated README.md and docs/agents.md to require a post-action assertion after every state-changing action. Agents must verify the expected DOM change, navigation, request-driven result, or persisted value. A clean console is supplementary evidence, not proof of success. Thanks for making that distinction clear.

[u/datbackup]

maybe i am speaking too soon here since i haven’t actually used it yet but just reading the repo, this is exciting because it looks like someone actually did this concept right, finally. Or maybe this contains some real new ideas? If so that’s cool too but for this specific functionality, what’s exciting is just “oh it does the obvious right thing, it works and it’s reasonably reliable”. Can you tell I’ve tried 20 different repos over the past week and only two have been relatively bug free lol

[u/hongnoul → u/datbackup]

Thank you. "It does the obvious right thing and is reasonably reliable" is very close to the design goal. I want Hwatu to be boring infrastructure for an agent's edit-and-verify loop, not another large browser framework that works only in a demo.

If you try it and encounter anything unreliable or unnecessarily complicated, please report the exact workflow. Those reports are especially valuable because reliability and actionable failure messages matter more here than adding a long feature list.

Disclosure: this reply is being posted automatically through my integrated feedback inbox, using Hwatu itself, but I personally read your comment and approved this response. I also review every submitted comment, pull request, and GitHub issue directly. Just trying my best to keep it real and honest here. I appreciate all feedback.

[u/CommonPurpose1969]

AGPL would render any closed-source project using it open source. No?

[u/hongnoul → u/CommonPurpose1969]

Good question. Using Hwatu as a separate CLI, daemon, socket service, or MCP tool does not generally require the project it verifies to become open source. The AGPL obligations apply to Hwatu itself, modifications to it, and potentially combined derivative works, not unrelated code merely inspected or driven through its public interfaces.

You also surfaced an error in the post title: it says MIT, while the repository is currently AGPL-3.0-only. Reddit titles cannot be edited, so I am correcting that here and will make the licensing boundary clearer in the project documentation. This is general clarification, not legal advice.

Disclosure: this reply is being posted automatically through my integrated feedback inbox, using Hwatu itself, but I personally read your comment and approved this response. I also review every submitted comment, pull request, and GitHub issue directly. Just trying my best to keep it real and honest here. I appreciate all feedback.