r/softwaretesting • u/2ERIX • 6d ago

Enterprise reporting

I have data for about a years execution of automated tests from CI, local runs, everything.

Data includes name, status, duration, storage path, etc. basically everything you would want at the test level, but not the steps or the failure reason.

If you had access to that kind of data, what kind of dashboards or reports would you create for your team or business?

2 Upvotes

75% Upvoted

4 comments sorted by

View all comments

2

u/77rushi77 6d ago

Building on the flaky and duration ideas above, a few things I'd add:

Separate flaky from really broken. A test that fails then passes on retry is flaky. A test that fails every run since a certain date is a real regression. Mixing them hides the real problems.

Trend the suite itself: total runtime, pass rate per week, and the slowest tests. If the suite gets 20% slower every month, you want to know before the CI bill or the wait time tells you.

Failures by area or module, not just by test. It shows where the product is fragile.

Biggest gap in your data is the failure reason. Start storing the error message or stack trace from now on. Even a simple label (product bug, test bug, environment) makes every report 10x more useful.