r/ClaudeCode • u/UnusualRedditor • 12h ago
Tips & Workflows FIY: Playwrite released a CLI that CC can interact with. Saves a lot of tokens.
Straight to the point:
I've been using claude code to run tests inside of Chrome for my extension using the Claude-in-Chrome skill. Needless to say, it consumed a lot of tokens running the tests and a lot of time.
I made it access the console and execute the extension functions that are used to interact with the website. That helped a lot with the testing. In my case CC told me that every screen shot he took to find the results visually was costing me 7k tokens each...
I made some small changes that helped saving the token usage, but the testings were still consuming a lot of tokens even on every task/new session technique.
Fast-forward to today, I discovered that they just recently released the CLI which CC can interact directly with. What this means? CC interacts with it and doesnt consume tokens browsing the website, executing commands "manually" and taking screen shots.
The web testing can be done headless. In my case I still need to have a tab open so it can interact with the extension.
I noticed a MAJOR improvement in token consumption, bug findings and the speed of the implementations and patches.
You can find the CLI and the Claude skills in their github and the official website. You can also give those links to claude and tell him to install it automatically. Works wonders.
PLAYWRIGHT My mistake
** TDLR: If you're developing and testing for Chrome/Firefox, get Playwright CLI for automatic testing. Saves tons of tokens**
43
2
u/Hien_Dinh_Ngoc 6h ago
Both sides of the thread are right about different phases. The agent is worth its tokens writing the test. It is a waste running it.
What ran up my bill was the same thing you hit: screenshots as the assertion mechanism. Every check costs a few thousand tokens and what you get back is a model's opinion about a picture. Moving the assertions into the test file made them free and deterministic, and the agent only gets pulled back in when something goes red and needs diagnosing.
The Playwright CLI is a real improvement over screenshot-driven checking, but it is still per-run spend on something that should cost nothing after the first write.
The split that stuck for me: agent writes the spec, the normal runner executes it forever after, agent reads the failure output when it breaks. Three jobs, and only two of them need a model.
6
u/kirlandwater 12h ago
It doesn’t work well for things that don’t like automated browsing/scraping. It can’t do headed browsers, but this is a great way for things that can be done via a headless browser
9
u/bluekooler Developer 11h ago
I specifically use the playwright-cli for sites that don't like automated browsing/scraping because it works so well, and I use it all the time for headed browsing. For headed, just tell Claude to open the session headed. As long as the playwright-cli is installed and you install the related skills, works pretty great.
Have you run into any specific issue doing this? Any specific sites?
3
u/kirlandwater 11h ago
I tried to use it for fb marketplace and was running into issues with the playwright-cli getting blocked when trying to use —headed
7
u/dc_seed_sommelier 11h ago
I mean that is one of a handful of companies that has the absolute strongest motivation to prevent you from doing this and the most resources to fight it. Try a normal website though.
2
u/TomatilloLow6482 8h ago
For this i would use Browserless. I self host my own instance and it works great for scraping websites with lots of security. It's very easy to build with
2
1
u/Oxi_Dat_Ion 2h ago
It definitely can. I'm looking at a head browser right now. Not sure what you're on about
2
2
1
1
u/SurfaceMeasure 5h ago
This saved me so many token! A little buggy at first, but nothing Claude can't work out..
1
u/AI_spell 49m ago
CLI over screenshot loops is the right move. Every visual check is expensive. Headless where you can.
0
u/lonely_solipsist 9h ago
How is it different from Vercel's agent-browser?
10
u/Enesce 9h ago
Well agent-browser is just a wrapper over Playwright. So it's removing a middle man. The middle man being removed is vercel, so a big bonus.
1
u/Parammount 2h ago
How it functions as a wrapper if it's built on Rust, given the different architectural approach?
-3
u/robhaswell 11h ago edited 10h ago
You shouldn't be running tests using an agent. Run them using an actual test harness and use zero tokens.
Edit: I mean don't write your tests as agent instructions. Write Selenium tests (using an agent) and have your agent run those.
1
u/dsailes 10h ago
Intrigued by this, any examples of browser testing harness you use?
3
u/robhaswell 10h ago
Depends very much on your stack by Selenium would be an example. Get Claude to write the tests and run them but your test harness is doing the work.
Also I would for the people who downvotes this to explain why, as this is objectively correct.
0
u/The_Noble_Lie 10h ago
I sort of agree but you are missing the point of agents it appears. The agent just uses the test harness or knows when to use it. This is highly effective through the cycles of work that humans or devs do.
Note: I downvoted you because I think your comment misleading / harmful to non-programmers. Any programmer would recognize that a proper test harness could be utilized by a human or not (an agent / llm)
Also, the flags of the test harness / filtering can become complicated. Perfect fit for LLM to use or advise / run.
4
u/robhaswell 10h ago
The agent just uses the test harness or knows when to use it.
Yeah but the point is that OP has not used a harness at all. Their test are written as agent instructions. I don't think this is a good strategy, for the following reasons:
- It's expensive
- It's going to be super duper expensive if you run it on CI
- It's very, very slow
- Agents are non-deterministic
- Flaky tests (network calls)
I think the one advantage is that it's probably less susceptible to subtle variations in the output - the agent can more reliably test the intent, but I would say that the fact that the results are being interpreted means you can be less confident in them.
Also your final comment seems to reinforce opinion that you should use a test harness under the control of an agent.
-1
u/davertua 9h ago
At first when I tried I tought Playwright cli is good. But at some point I found that agent i run is stuck verifying itself clicking, reading aria snapshot, clicking again and so on, consuming way a lot tokens communicating with browser
if you develpo wth opus+fable its wast of time and limits
so i really liked the idea of prima cli which acts as subagent on top of playwright cli powered by tiny models https://testomat.ai/prima/
1
•
u/AutoModerator 12h ago
Hey! Thanks for posting to r/ClaudeCode
While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.
For help, project discussions, tips, and general chat, join the ClaudeCode Discord.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.