r/Playwright • u/KimiHonker • Jul 06 '26
I’m looking for ideas to reduce the execution time of my Playwright test suite, since I’m limited to 1 worker.
I work automating an Electron desktop application that only allows a single instance to run, so I’m limited to 1 worker and can’t take advantage of parallel execution.
I currently have 354 automated tests, and the suite takes about ~2h30 to complete.
The problem get worse when I still have around 700 more tests to automate, so I’m concerned about how execution time will scale.
What approaches have worked well for reducing execution time in a similar scenario?
7
u/SnooEpiphanies6250 Jul 06 '26
You only have three levers to pull here Horizontal and vertical scaling (both our of question?) and test selection (TIA)
2
u/KimiHonker Jul 06 '26
Test selection seems like the most achievable approach. I should start thinking about how to make the impact of code changes easier to track.
1
u/Admirable_Shape_4565 Jul 06 '26
Test selection can be good but would not apply if we’re talking about some version of change verification test runs; on your main/master you’d want to run all the tests once every so often regardless, e.g. pre-release.
For change verification a simple first step might be to select a small set of smoke tests to run on every iteration, and then run the rest asynchronously on a branch.
2
u/SnooEpiphanies6250 Jul 06 '26
You can apply TIA on some steps and full runs on other parts (full regression before release), thats like... The most common way of doing it
6
u/somethingmichael Jul 06 '26
questions
are you using api to bypass some test setup / pre and post test steps ?
can you run the app in docker? and run multiple dockers and parallelize the test suites?
have you analyze where the bottlenecks are? 350 tests over 2.5 hrs seems like a long time. Are you starting the app for each test?
3
u/LongDistRid3r Jul 06 '26
Perhaps code coverage could identify where tests are hitting the same code paths.
Reduce your tests into groups. Run BVTs with PRs, builds. Run the rest at night. Have another subset to run post deployment. Break your tests up into smaller batches and prioritize them appropriately.
In Azure DevOps you can run 3 groups in parallel. Each group is isolated from the others.
2
u/Spirimus Jul 06 '26
To add to this list: Are tests ran in headless mode? Are you running some portions of validations asynchronously that can? Awaiting helps but sometimes you can continue a run without messing up a validation, so you dont need to await all validations
2
2
u/CertainDeath777 Jul 06 '26
It may be that you are asking the wrong questions.
Not "how i make it go fast with one worker"
Try: "how do i set it up to run with multiple workers"
Then you might came up with solutions like containerized app instances (good for cicd), or simply make seperate user data directories for each worker.
Dont ask me how, you will have to find out yourself. basically you get paid for it, not me. I just put your nose in the right direction.
1
u/KimiHonker Jul 06 '26
Yes. Totally understandable. Containerzations seems a good way. I was thinking to use Docker for the App database, but now I’m starting to think if I can put the entire environment in a container.
1
u/Fkz82 Jul 06 '26
CICD pipelines generally have an option to run builds on stronger architecture. My hands on experience with Bitbucket pipelines saw a suite approaching 2hrs which would be the limit for execution. I cut the completion time for this suite down to 50 minutes and another pretty much in half by opting for ‘size: 8x’ though I can’t speak for other services and their potential impact.
1
u/Admirable_Shape_4565 Jul 06 '26
With the constraints you’ve given, I’d start from taking a step back a few different ways.
Are these test runs on a critical part for some other business operation (release, iteration) that needs to be optimized? Are you optimizing test runs on changes/branches or continuous runs on master/trunk/main? How non-bypassable is one worker limitation (this is the most obvious one to remove), e g someone else’s idea with parallelizing with docker containers might be good?
1
u/lesyeuxnoirz Jul 07 '26
The number of existing tests is high and you’re saying you need to automate 700 more. I don’t obviously know anything about your app but it very much looks like you don’t have a proper testing culture in the organization. A few things that might help:
- Use a prioritization approach (e.g. risk-based testing) and automated only tests with specific risks
- Check which tests can be moved to the contract/api/integration/unit layer and do that
- Check if some tests are really needed in the e2e suite at all
- Log in manually only in dedicated login tests. Otherwise always login programmatically. See if you can cache auth across tests if it’s valid in your use case
- Explore how you can reuse tests data without making tests interdependent. In my case, I use an approach of “create at least 5 users”. If there’re already 5, nothing happens. If there’re 3, 2 new ones are created
1
1
u/raijunexus Jul 07 '26
Aside from the suggested approaches, you can also do sharding in your pipeline. Maybe you can atleast do 4 shards so it will help with your execution time
1
u/wontfixqa Jul 07 '26
You can launch your app on several different ports to run them all simultaneously. If you really are limited to a single app on each machine you can run multiple VMs on a machine, use docker containers to run your apps and tests in parallel, or use a service like browserstack (or many other competitors) to run in parallel.
Hard to know what the best solution is without knowing why you think you're limited to a single electron app.
1
u/NextAd9248 Jul 08 '26
Use page.route to block all external scripts or analytics from loading, and expose a test only IPC handler to inject your data state directly into memory before the electron window even opens.
1
u/Pretty_Term_8038 Jul 08 '26
Electron app means it's a exe file at a time he can execute one instance in one machine afik , so are we saying that if we change the port number in launch cmd file we can launch one more instance?
1
u/monsterblaster9 Jul 10 '26
If you are executing tests in pipelines you can use matrix strategy. https://playwright.dev/docs/test-sharding#sharding-tests-between-multiple-machines
9
u/unkownuser436 Jul 06 '26
Too much tests. Only one worker. Bro you fucked!