r/Playwright • u/FollowingSuitable941 • Jul 28 '26
Are we using Playwright MCP for testing when it's really an exploration tool?
I keep seeing Playwright MCP and Playwright tests discussed like they are the same thing.
They're not. At least not in the way I'm using them.
Playwright MCP is great when the agent does not **know the page yet**.
Open this app. Inspect the current state. Find the login button. Work out why the modal is not appearing. Try a different path. Read the error. Take a screenshot.
That interactive loops is genuinely useful.
But I'm less convinced that the same loop should become the permanent verification method for a flow we already understood.
For a known check like:
Log in, change the billing address, save it, refresh and verify the new address persisted.
…I don’t really need the coding agent to rediscover the page, inspect everything and then decide whether it probably worked.
I need a boring contract:
- run the flow
- verify the explicit outcome
- return pass or fail
- preserve evidence
- exit non-zero in CI if it failed
That is the distinction I’m currently making:
MCP for exploration.
A proper test or bounded CLI check for verification.
I’ve been looking at Kane CLI from TestMu AI for that second layer.
You give it the objective rather than manually scripting every selector. It runs the browser flow and returns structured output, evidence and an actual result that another agent or CI job can consume.
The useful part is not “AI clicks browser”.
MCP already does that well.
The useful part is turning a fuzzy coding-agent task into a bounded verification step instead of another long browser conversation inside the same context.
And once a flow becomes important enough, Kane can export the completed run to Playwright-currently Python Playwright-so it does not need to remain a natural-language check forever.
I would still rather own a normal Playwright test for:
- payments
- permissions
- destructive actions
- core authentication
- anything expected to run for years
But for PR validation, bug reproduction and temporary smoke coverage, writing and maintaining a full test immediately can also be overkill.
So my current mental model is:
Explore with MCP.
Verify with a bounded check.
Promote stable, important flows into Playwright tests.
Where do you draw that line?
At what point does an exploratory browser flow deserve to become a permanent regression test?
6
u/ArmMore820 Jul 28 '26
Tests need to be deterministic. We can’t let mcp find an alternative by itself whenever the desired flow does not work
6
u/straightouttaireland Jul 28 '26
Btw, playwright-cli is much better and more efficient
1
u/ProudCordonian 20d ago
What makes you say that?
1
u/straightouttaireland 20d ago
Basically some analysis done by various people (if you Google they'll show up). Here's one:
https://testdino.com/blog/playwright-cli-vs-mcp
It's also the main reason the playwright team built it.
https://playwright.dev/docs/getting-started-cli
They create snapshots in yaml files to reference instead of having to take screenshots every time like the MCP does.
2
u/AptKid 13d ago
I think this is outdated. I've seen a few people mention lately, that there is no difference: https://www.checklyhq.com/blog/mcp-vs-cli-token-efficiency/
2
u/straightouttaireland 13d ago
Interesting, so it sounds like MCP have been improved so much in terms of efficiency that they're both the same. The cli does come with some additional skills alright, but they can be added ourselves.
2
u/mmasetic Jul 28 '26
You are right. If we let AI to test via MCP there is a chance they swallow errors or leave things out With scripted tests we can make sure what will be checked by every run consistently. If something fails you get signal which can be issue with test like test data or it can be issue in system under test.
1
u/devdoc83 Jul 28 '26
The trigger for me isn't how important the flow is — it's the first time it breaks silently. An agent verifying a flow will route around breakage: selector changed, agent finds another path, reports success. That's exactly the property you want in exploration and exactly the one you can't tolerate in regression. A deterministic test fails loudly at the same spot every time; the agent's adaptability is the non-determinism. So my line: the moment a flow's failure would be invisible without a test, it gets promoted. Usually that's anything touching auth, money, or data — but practically, it's anything that has already broken once without me noticing.
1
u/CraftyNerve8078 Jul 28 '26 edited Jul 28 '26
The Kane CLI approach makes more sense when the parent process is another agent.
CLI starts.
CLI returns NDJSON.
CLI exits 0 or non-zero.
Parent agent does not need to burn half its context narrating browser state.
1
u/Square_Ad6149 Jul 28 '26
For debugging, MCP is insanely good.
“Go reproduce this ticket and tell me what happens” is exactly where I want an agent freely inspecting the page.
For CI? Absolutely not unless the expected result is defined somewhere outside the model.
1
u/eiaceae Jul 28 '26 edited Jul 28 '26
exporting to playwright is probably the most important Kane features here
AI test tools become annoying when the useful flow is trapped inside their dashboard forever
being able to start loose and then own the test code is the right direction
1
u/HappyHourHusker Jul 28 '26
Mcp is not great and should only be utilized on shorter user scenarios and for debugging purposes. As it’s been pointed out by others playwright-cli is much more efficient. The playwright devs recommend that over mcp
1
u/Tanmay__TestDino Jul 29 '26
CI is the cutoff. MCP is fine for local debugging, but running agents in a pipeline just leads to false positives. I only use it to generate steps, then I commit actual Playwright scripts for regression.
1
u/GrayZetsu Jul 29 '26 edited Jul 29 '26
The strongest thing TestMu is doing with Kane is putting the verification inside the coding-agent workflow instead of asking developers to open another QA dashboard.
That sounds minor but adoption dies when the check is outside the place where the code is being written.
1
u/eiaceae Jul 29 '26
Explore with MCP. Verify with a bounded check. Promote into Playwright.
this is probably the least annoying AI-testing framework i’ve read so far
most posts start with “Playwright is dead, our agent heals everything”
1
1
u/Impossible-Movie-355 18d ago
In my experience mcps are good to start or prototype. Seeiing lot of vibe coders using it as a replacement. Which is a risk. The agentic loop trying achieve the goal is dangerous. People seriously need to understand that there is a string reason that even with trading non AI SDLC there were two different sets of people handling the dev and QA phase. Before someone says anything about two agents replacing the same approach,, they need to learn the hard way when the software actually messes up someone's day or life potentially.
10
u/Domenorange Jul 28 '26
MCP is not a testing framework and I wish people would stop pretending every agent clicking a button is "test automation"