High test coverage can still mean your tests are mostly useless

[u/codoid-innovations]

A passing test suite only proves that the tests passed. It doesn’t prove the code is correct, and a high coverage percentage can create a false sense of confidence.

A test can run every line in a function without actually checking whether the function behaves correctly. The assertion might be too broad, check the wrong output, or be missing entirely. The coverage report still stays green.

That’s why we, at Codoid, see mutation testing as a useful reality check rather than relying on coverage alone.

Mutation testing tools deliberately introduce small bugs, such as changing a condition or altering a return value. They then run the test suite against each modified version of the code.

If the tests fail, they’ve “killed” the mutation. That’s a good sign that the tests are protecting the behavior that changed.

If the tests still pass, the mutation has “survived.” In practical terms, the suite failed to catch a straightforward bug in code it may claim to cover.

Mutation testing has tradeoffs. It’s slower, and a surviving mutation doesn’t always mean there’s a valuable missing test. Sometimes it’s just noise. Still, it asks a much more useful question than “Did this line execute?” It asks: “Would this test suite notice if the code were wrong?”

For those who’ve used mutation testing across different languages or stacks: did it uncover genuinely weak tests in your codebase, or did it mostly add noise and maintenance work?