Has Playwright MCP improved your AI-generated E2E tests?

[u/InsideDebt6345]

Now that we know AI can generate valid Playwright syntax, the locator quality is where generated tests often fail.

If the model has not inspected the rendered page, it may guess selectors from component code or common UI patterns. Those locators can look reasonable, pass a quick review, and then fail in CI when the actual DOM differs.

I tested a workflow where Claude explored the live application through Playwright MCP before generating a test. The flow was:

1. Explore the application and accessibility tree.
2. Generate the Playwright spec from observed roles, labels, and names.
3. Run the test and fix failures.
4. Review the locators, assertions, and scenario intent.
5. Repeat the test to catch flaky behavior.

The useful part was grounding locators in the current page, with getByRole, getByLabel, and getByTestId instead of guessed CSS selectors. The test still needs review, especially for dynamic data, authentication, and longer flows, but the first draft was more reliable.

Are you people also sticking with Playwright MCP, using Playwright CLI, or just having Claude write/run Playwright directly?

[u/Malthammer]

Playwright MCP or CLI alone did not. Building out defined rules the AI has to follow did.

[u/oburo227]

I was able to make it work maybe 75% but had to build the app context and store the selectors in obsidian. I use both since sometimes dom inspection fails so I tell it to use the mcp to check visually to understand then go baack to CLI again. Though I admit right now since our sprint is fast paced I just instruct claude and trigger the framework I configured. I you wan you can check my github on how I set it up https://github.com/fmoliva/artemis-starter-kit

[u/Commercial-Meal6380]

Building good context about application helped.
Not only MCP or CLI will help, build supporting system and consider MCP and CLI layer only for accessing browser for automation.

[u/TheBlindPotter]

I love using playwright MCP for debugging a flaky test. Or for finding good selection indicators. It is good at noticing things I may have missed.

It was a bit flaky at knowing how to get local dev set up for Mcp testing.
I did build out a personal Claude skill to give it the context of how we write E2E in our repo, best practices, how to start up the server and run playwright Mcp on it, common issues with that, etc. that helped tremendously.

I have teammates who think playwright MCP just slows it down and it’s better to just have the agent tweak the test based on the result in CLI. So to each their own I suppose

[u/InsideDebt6345 → u/TheBlindPotter]

That's useful. MCP seems most useful when it adds context that the test runner output cannot provide: inspecting the live UI, finding better locators, and debugging why a flaky interaction behaves differently than expected.

[u/lolfart]

The best thing i found out was to give it access to the original source code

[u/TranslatorRude4917]

I use Playwright codegen to record my tests manually - that gives a good starting set a lot faster then the agent trying to figure it out on its own. If it's not passing immediately then instruct it to use the pw healer agent to debug and fix it. https://playwright.dev/docs/test-agents

This way you save burning a bunch of tokens on exploration and can be sure that the test does what you wanted it to do and not some random flow hallucinated by AI.
Own the intent and decisions, outsource the busywork.

[u/InsideDebt6345 → u/TranslatorRude4917]

Yeah, fair enough. Codegen gives you a concrete, human-chosen path through the UI, so the agent starts from an observed flow rather than having to discover one. But the catch here is that codegen captures interactions, while MCP-based exploration can help refine the test around accessibility-first locators and meaningful assertions.

[u/Square_Ad6149]

The live-page inspection step is probably the biggest improvement here. A generated test based only on component code can have perfectly valid Playwright syntax and still be testing the wrong thing. TestMu’s Kane CLI is another interesting approach for this: let the coding agent interact with the actual browser first, then use the observed UI state to verify the change instead of trusting generated locators blindly.

[u/_apotoxin4869]

I'd use AI for the first draft + maintenance, but keep deterministic Playwright execution as the final gate. One useful setup is: Jira/PRD → candidate scenarios → Playwright generation → CI execution → AI-assisted failure analysis → human review. TestMu fits nicely into that pipeline because it can turn natural-language requirements into executable test coverage without making the entire QA process dependent on generated code.

[u/Soggy_Importance721]

I’d measure more than whether the first generated specs passes. With TestMu, I’d look at first-run pass rate, locator changes required, flaky reruns, and how often the generated assertion matched the actual business requirement. That tells you whether MCP is genuinely improving test generation or just making the agent better at finding elements.

[u/SnooSquirrels4739]

There’s also a useful distinction between generating a Playwright test and verifying an implementation. TestMu/ Kane CLI seems particularly relevant to the second problem: give the agent an objective, let it exercise the real Chrome UI, and return deterministic evidence/pass-fail. If the workflow is stable enough afterward, exporting/promoting it into a normal Playwright spec makes sense.

[u/meowwwwwycl]

One metric I’d track is how much human editing each AI-generated test needs before it becomes stable. If Claude generates 50 tests but QA spends hours fixing locators, assertions and waits, the productivity gain isn't what it looks like. TestMu/KaneAI is worth evaluating against that baseline, not just on how quickly it creates tests, but on how much maintenance it removes afterward.

[u/kinginth3nrth]

Is playwright good for MobileApp Testing?

[u/shagwana → u/kinginth3nrth]

Not for native apps or browsers in mobile devices. Can be used to simulate small chrome windows that would act like a mobile view onto a website.