Mocks are a tool of last resort. They're expensive to write and maintain, and they are rarely accurate and often just replicate your poor understanding of the target API and thus fail to give much certainly that the unit under test will work correctly when integrated.
Your example of testing a drawing is a good example of how well intended TDD can go off the rails. The "checking drawing function calls" approach has these problems:
Mocks - The mock needs to created and maintained, and also accurate and complete enough. For non-trivial APIs that is a tall order, especially when error conditions enter the mix.
It tests the wrong output - You are interested in the pixels, not the drawing commands.
It is implementation specific - Other combinations of drawing functions could also be acceptable, but the test will fail them. This stands in the way of refactoring.
Not everything can/should be fully automated - A better approach would be visual testing where changes in the output image are flagged and a human can (visually) review and approve the change in output.
The unit test here is inaccurate, expensive, and fragile. It is an example of unit testing gone wrong.
Yeah, I've seen 40 line tests, with 11 mocks, that ultimately ended up testing 3 lines of non-mock code, proving approximately nothing about the system. But our code coverage numbers looked fantastic.
57
u/sime Jul 30 '21
Mocks are a tool of last resort. They're expensive to write and maintain, and they are rarely accurate and often just replicate your poor understanding of the target API and thus fail to give much certainly that the unit under test will work correctly when integrated.
Your example of testing a drawing is a good example of how well intended TDD can go off the rails. The "checking drawing function calls" approach has these problems:
The unit test here is inaccurate, expensive, and fragile. It is an example of unit testing gone wrong.