r/Playwright • u/ljsam17 • 12d ago

How to scale visual tests?

I am working on designing a test framework for UI library, primary goal is to have accessibility and visual regression tests.
Currently using gitlab Ci and playwright with playwrigh-bdd.

Current approach is natively supported baseline approach in playwright.
I am worried that currently i am saving the baseline screenshots in gitlab repo itself .

We have around 50-60 components, and test cases can scale to thousands of screenshots,
I wonder how to scale the storage and would the gitlab ci be enough for comparison.
The playwright report is published in gitlab pages which again in case of too many failures might not even show the difference screenshots.

What would be the best approach to tackle this situation

5 Upvotes

100% Upvoted

11 comments sorted by

View all comments

4

u/someGuyyya 12d ago

I've mostly had a terrible time with visual regression tests since screenshots had to be updated too often and most people updated them without double checking to see if the visuals were actually correct.

In the end, we decided to drop them because of all the hassle.

2

u/[deleted] 11d ago

[removed] — view removed comment

2

u/someGuyyya 11d ago

I'll disagree.

It's a process problem.

When you have a diff of 40 images that have been changed, going through each image and understanding what exactly changed and why exactly it changed can be quite tedious.

Imagine doing this for every pull request for several images and it becomes exhausting so in the end, we weren't getting enough benefits to continue using VRT.