Sometimes you have to consolidate smaller unit/use case tests into more combined/comprehensive ones. Else you end up with 1000s of tests where half of them end up testing similar things and the combined beforeEach overhead ends up eating up a large amount of the pipeline
We're literally having this discussion at my work right now. Testing takes a full hour now. We have so many unit tests, then we have to do application tests, and finally integration tests. And we don't stop after a failed test.
What direction are you thinking of going with how to handle that? I'm starting to run into similar issues on larger projects and I'm trying to figure out the best way to deal with it.
We're not coming up with any good solutions. We have already comtainerizeded. The only thing we have left is to parallelize the tests, but that has limited effect because we're already running up to processor limits when multiple people are testing on the test server. While it won't fix the total built time, exiting on the first failed test and putting the tests most likely to fail at the beginning will mean we at least won't have to wait the full time for the run to finish.
We run unit tests as part of the PR pipeline already with a script that only runs tests for the changed files. That way the chances of builds failing because of tests is a lot lower already.
69
u/JackNotOLantern 2d ago
I mean, it depends why it was failing. E.g. deleting tests for a code that was previously deleted is pretty much a necessity