I admit that it was only relatively recently that I started taking a stricter approach to developing and refactoring tests. It took me a while to realize the enormous methodological return of moving from a console application for debugging (…) to having a complete, robust test suite properly in place. In fact, these days I only see through tests. I mean, when I develop new production code, I rarely step through it in a debugger if the corresponding tests indicate everything is working well.
(Update: in 2019, I published a new work on agile practices that help us write better-designed, higher-quality code—and therefore better tests: The Practical Agile Programmer’s Book.)
However, without oversimplifying how easy it is to spot other people’s mistakes, I often see (far more often than I’d like) that even when an architecture and methodology are in place that allow and require testing, tests are written just to get by, covering what is usually called the happy path—what I personally call “happy tests.”
What is a happy test? When we write a new class or add a feature to existing code, we naturally tend to write tests that we know, consciously or unconsciously, will cause few problems. We test the best-case scenario, the one our code handles well: the happy test tells us our code does what we built it to do. But if we limit ourselves exclusively to happy tests, we ignore much of the purpose of having a test-backed solution. We focus only on happy tests because we see writing tests as a necessary evil rather than a fundamental pillar of professional, high-quality software development.
The goal of writing tests isn’t merely to have a mechanism that automatically checks that something works now and will keep working in the future. We don’t write tests only for the ideal situation (the happy path); we should also develop them to check all the difficult situations that could compromise a particular feature. We sometimes forget that tests are meant to detect errors. If we stick to the simplest cases, we’ll probably hide bugs that will eventually blow up in our hands.
When is the best time to detect errors and problems? While we’re developing a new feature, or after the software is already in production and one or more clients are using it? The answer is obvious.
Recently, I developed a Windows service to instantiate certain web services that emulate the behavior of a type of device. One of these devices is simulated by instantiating its corresponding web services on a specific port. Naturally, I wrote tests to ensure the web services start correctly on port 8000. Is that all? Absolutely not. What if that port is already in use when the software is deployed on a production machine? In that case, the tests need to ensure that we’re properly notified when this happens, that an error of this kind doesn’t crash the entire Windows service, and so on. If I had stopped at the happy test, I might have finished sooner, but I would almost certainly have had to spend much more time fixing those errors in production with an unhappy client…
This is just one example, but sometimes we don’t realize how much effort and time we can save by catching errors before production simply by creating a proper test suite during development. Few things are more reassuring in software than knowing your product has been tested well enough that deployment problems are very unlikely.
A law of software might be this: the more and the better you test software, the greater the return, in terms of spending less time debugging, having fewer clients report problems, and so on.
This isn’t about creating the greatest possible number of tests for its own sake, but about creating tests of sufficient quality that cover all possible cases.
I’m afraid this is something we often underestimate. Creating and writing sufficiently good tests is not trivial. The problem is that in our profession we often have to be generalists (which means doing a bit of everything). This isn’t a simple subject: software testing is a distinct role for a qualified person with specific professional skills, as this excellent book, Software Testing, makes clear.