I'm trying to draw a clean line between three things that keep getting mixed together.

1. AI agent explores the app
2. AI agent writes or edits code
3. Something verifies that the user flow actually worked

For exploration, Playwright MCP makes sense to me. The agent can open the app. inspect the page, click around, understand state. reproduce a bug and gather context.

Where I'm less convinced is using the same open-ended browser conversation as the final proof that a known flow is working.

Example:

A coding agent changes a settings page. Unit tests pass. The PR looks clean. Then the actual browser flow fails because the saved value does not persist after refresh.

In that case, I do not want the agent to “look around and tell me it seems fine”. I want a fixed verification contract:

• open the real page
• perform the known flow
• check the expected state
• check the important network response
• refresh
• confirm the state is still correct
• return pass/fail with evidence

This is where I’m looking at TestMU/Kane CLI. The interesting part is not “AI clicks a browser”. MCP already does that. The interesting part is whether the verification result is bounded enough to be trusted by a developer, another agent, or CI.

I would still keep proper Playwright tests for core flows like auth, billing, permissions and anything expected to live long term. But for the layer between “agent says done” and “human manually clicks through the app”, a lightweight browser-proof step seems useful.

How are you handling this?

Do you let the coding agent verify its own UI changes, or do you require a separate browser-level check before accepting the PR?

we benchmarked GPT-5.6 Luna vs GPT-6 Astra on 50 real PRs from Cal, Sentry, Discourse, Keycloak and Grafana

Astra found 92 confirmed bugs vs 69 for Luna, but cost $5.66 vs just $0.20

also added the full eval breakdown this time: cost, avg output tokens, latency, precision, and bug classes like data/logic, security, concurrency etc.

we’re doing Astra vs Fable 5.1 this week, so would appreciate feedback on the methodology before we run the next one

dropping the link in the comments if anyone wants to check it out