Why I Write Automated Tests
25 January, 2002
I'm still mystified why so many programmers don't write automated tests. Instead they rely on ad-hoc, manual testing. While many others have outlined why you should write and maintain automated tests, here I'll give you my reasons for writing automated tests. You can decide whether or not it is right for you. (My way is of course right, but I'll let you decide whether you want to write software the right way or the wrong way.)
So That I Know Whether the Code Works
Firstly, I write tests so that I know whether or not the code works. To me, this is important. Why write something that I have no way of knowing whether it works?
Sure I can just use the software and observe whether it works. But that takes a lot of work, and I've got better stuff to do. (Like skiing, or -- in the summer -- biking.) Instead, I prefer to be able to run a script that tests the software to make sure it works. Or push a button in the GUI that runs a bunch of internal tests.
Of course, there are situations where automating a test is more work than just testing something by hand. Especially when failure is subjective (like the placement of elements in the UI). Or when it involves usability -- I don't know of any way to automate usability testing.
The most important part is to have automated tests, that you can easily run, for all of your classes, modules, packages, subsystems, whatever it is that you write for low-level units. This way you can at least know whether all of the components in your software work properly.
If the system you're building lends itself to running scripts or performing easily verifiable external actions, you should also have automated system tests. It just makes life easier.
So That I Know When I'm Done Coding
I write test-first. That is, I figure out what I have to do before I write a line of product code. Then I figure out how I can observe the behavior that I need to add. Then I write the test, run the test, and watch it fail (because I obviously haven't written the product code yet).
Now I'm ready to start writing the product code. The advantage here is that I know the minute I'm done -- the test passes. If the test is written properly, then once it passes I'm done. Why bother adding code if what you've got passes all of your tests? Given that your test codifies the needs of your customer, you're not adding any value by adding more code. Move on to the next task -- that is the next customer requirement, which is where the value lives.
If you find that you've still got to add code even though what you have passes the test, then you haven't written the test properly. It is obviously incomplete. This is not to say that you can't code in small chunks. I do this all the time. It's just that once I pass the test, I check in. Then I check back out, write another test, and start coding again. The code that is checked in always works. (At least when I'm playing by the rules. Sometimes temptation still leads me astray.)
So That Bug Fixing Doesn't Introduce New Bugs
Anytime you mess around with the code, there's a certain likelihood that you're going to break something. This is not to say that having tests is a permit to have fragile or sloppy code, but that likelihood does exist and it helps to have a tool to prevent regressions.
Oops. There, I've said it. If you have automated tests, you (by definition) have a regression test suite. This prevents you from breaking functionality that is already in the software when trying to fix a bug.
To prevent the re-introduction of a bug that has already been fixed, (you guessed it) write a test that exposes the bug. Then fix the bug, watch the test pass, pat yourself on the back, and check in the fix. If you (or someone else who comes along later) has to fix another bug in the same part of the code, you have a vastly reduced risk of re-introducing this bug.
To Support Refactoring
Refactoring is a technique to restructure code in a disciplined way. That's a direct quote of the definition from the linked website. (Why bother putting it in my own words when Martin Fowler has done it quite succinctly already?) To elaborate a bit, refactoring uses formulaic conversions of code to fix common problems ("smells" in the lingo of refactoring) in the design and structure of code.
As you might guess, it involves a fair amount of shuffling code around: renaming variables and methods, moving methods between classes, moving code between methods, etc. If you have the book, you know that most of the code transformations are fairly mechanical. You also know that the phrase "compile and test" appears at least once in most of the transformations. Having automated tests that you can quickly run are a prerequisite to performing effective refactoring.
All My Tests Are Version Controlled
This last point isn't so much about why you should test, but more a tip about the "how". When I write a chunk of software (subsystem that is part of a larger project, small tool, module that is part of a larger tool, whatever), I create a separate file that contains the tests. It is basically part of the code for the product. That means that this file gets checked into version control with all of the other source files. It also means that when I get a bug report about a released version of the product, I can go back and get the version of the tests that goes along with the version of the code that went into the product release in question. I can then update the tests to expose the bug, fix the code, and check everything back in . When I merge that bugfix forward to my next release, any new tests will also get merged forward.
So there you have it -- all the reasons that I write automated tests. It basically all boils down to speed, reproducibility and predictability.