"Brittle" Unit Tests

I have a friend who is new to unit tests and therefore has never done Test-Drive Development. His management has decided that developers should now write unit tests. But there has been some debate about what should be tested. Since his "application" is really a class library, a lot of the functionality is hidden inside the library (the public API is just a small part).

Some of the people he talked to suggested not writing unit tests for the implementation (just test the public APIs) because they were nervous about making the library "brittle". As my friend said, "they define brittle as in it's really easy for someone to make a change and break stuff."

The comment about brittle unit tests is actually very interesting. I’ve encountered that myself. It’s frustrating when I have to go back and update unit tests after doing a large refactoring. It certainly makes the unit tests “feel” brittle. But I think this feeling is a result of changing my approach to development rather than a real issue. I think it can be broken into two cases:

1. Unit tests fail, showing bugs in my code. This is obviously the “good” case.
2. Unit tests fail because assumptions have changed. This makes unit test feel brittle. But actually it’s a good thing. They fail because my assumptions or “specifications” have changed. That means I really do need to rewrite the tests to reflect the new specifications that resulted from design changes.

There is another way to look at item 2 above. Often the broken unit tests are a result of bugs in the specifications, so what you're really faced with is having to interrupt what you were working on to fix bugs. This can certainly be unsettling, because it's a brick wall you have to scale before you can continue with your thread of thought. However, since you're fixing bugs as you go, your code will be much more reliable.

Unit Tests for Learning and Experiments

Very often when I'm learning a new set of APIs, I like to write some test code to try things out. In the past I would create a new Windows Forms project and put some buttons into the form. Then I'd put my test code into the Click event handler for the buttons and use the debugger to step through the code and look at the results.

Not anymore! Now that I've gotten used to using Test-Driven Development to write code, I now write the small learnings as unit tests, or as a unit tests that call some test code in my application. Since unit tests are run without starting up the entire application, they run very quickly.

I put this into practice just today working on some code that uses the XmlSerializer to serialize some of my objects. In this case my unit test is very simple:

[TestMethod]
public void ShouldSerialize()
{
    MapNode first = new MapNode("first");
    MapNode second = new MapNode("second");
    MapNode child = new MapNode("child");
    first.Nodes.Add(child);
    MapNode[] nodes = new MapNode[] { first, second };
    FolderViewpointItem item = new FolderViewpointItem(10, nodes);

    XmlSerializer xml = new XmlSerializer(item.GetType());
    StringWriter writer = new StringWriter();
    xml.Serialize(writer, item);
    Assert.Fail(writer.ToString());
}

Since I simply wanted to see the XML generated, I used the Assert.Fail method to display the XML as the message shown in Visual Studio's output window. Very fast, very simple. And I can use any of the classes in my application to try things out.