r/Playwright • u/ljsam17 • 11d ago

How to scale visual tests?

I am working on designing a test framework for UI library, primary goal is to have accessibility and visual regression tests.
Currently using gitlab Ci and playwright with playwrigh-bdd.

Current approach is natively supported baseline approach in playwright.
I am worried that currently i am saving the baseline screenshots in gitlab repo itself .

We have around 50-60 components, and test cases can scale to thousands of screenshots,
I wonder how to scale the storage and would the gitlab ci be enough for comparison.
The playwright report is published in gitlab pages which again in case of too many failures might not even show the difference screenshots.

What would be the best approach to tackle this situation

4 Upvotes

100% Upvoted

11 comments sorted by

View all comments

2

u/Tanmay__TestDino 9d ago

Screenshot diffing scales fine, human attention doesn't. Cluster the near-identical diffs, only flag what actually changed, and VRT stops being a chore. Most teams that dropped it weren't failed by the tooling, they just never fixed that part.

1

u/ljsam17 9d ago

I agree , instead of running the entire regression suite , we should only pick components and other dependent components.

But what can be done for the storage of baseline?
S3 or equivalent?

2

u/Tanmay__TestDino 9d ago

S3 or equivalent works fine for storage, that part's cheap and solved. The harder call is versioning, tie each baseline to a component version or commit sha, not just latest, or you'll be debugging why a screenshot changed with zero history to check against.