"Brittle" Unit Tests

I have a friend who is new to unit tests and therefore has never done Test-Drive Development. His management has decided that developers should now write unit tests. But there has been some debate about what should be tested. Since his "application" is really a class library, a lot of the functionality is hidden inside the library (the public API is just a small part).

Some of the people he talked to suggested not writing unit tests for the implementation (just test the public APIs) because they were nervous about making the library "brittle". As my friend said, "they define brittle as in it's really easy for someone to make a change and break stuff."

The comment about brittle unit tests is actually very interesting. I’ve encountered that myself. It’s frustrating when I have to go back and update unit tests after doing a large refactoring. It certainly makes the unit tests “feel” brittle. But I think this feeling is a result of changing my approach to development rather than a real issue. I think it can be broken into two cases:

1. Unit tests fail, showing bugs in my code. This is obviously the “good” case.
2. Unit tests fail because assumptions have changed. This makes unit test feel brittle. But actually it’s a good thing. They fail because my assumptions or “specifications” have changed. That means I really do need to rewrite the tests to reflect the new specifications that resulted from design changes.

There is another way to look at item 2 above. Often the broken unit tests are a result of bugs in the specifications, so what you're really faced with is having to interrupt what you were working on to fix bugs. This can certainly be unsettling, because it's a brick wall you have to scale before you can continue with your thread of thought. However, since you're fixing bugs as you go, your code will be much more reliable.