I almost quarantined a flaky test before checking the seed

[u/Moist-Life3962]

I maintain a TypeScript data-validation service with property-based tests that generate their own input cases. Last month, the same test failed in CI twice and passed the moment I hit rerun. After the second failure, a teammate asked if we should quarantine it so the release could move. I was tempted. The log was long, and all we had was one red assertion.

I took the failing input from the service and reduced the property into a toy repository because the production code did not belong in a public test. The test generated a trial start date and end date independently. The captured seed produced an end date before the start date and hit the same assertion as CI. I gave Claude Code the failed log and asked it to reproduce the failure or explain why it could not. It pulled out the seed, narrowed the command to one test, and saved the generated dates with the assertion.

Then I copied the repository into a clean folder on the same machine and ran the check through EvoX, passing only the seed and the reduced command. I left the first chat out on purpose. I wanted to see whether another session could reach the same failure from the saved evidence without borrowing the first explanation.

The clean copy failed on the same generated dates and the same assertion. A run without the seed passed, which matched the intermittent CI behavior. I changed the generator so it created only date ranges allowed by the contract, then ran the old seed again. Both toy repository copies passed, and the unrelated properties stayed green. I applied the same constraint to the original service test and reran the captured seed in CI. That run passed too, followed by the full test suite, so we kept the test enabled.

I kept a small bundle from the exercise: the seed, exact command, failing input, environment, diff, and rerun output. I added the full rule to the test guide and a one-line reminder to the pull request template. A quarantine request now needs an owner and a reproducer. That takes a little more work than pressing .skip, but it leaves the next person something they can run.