I’ve been an ardent supporter and practitioner of TDD for just about 20 years. I was first introduced to Test-First Programming in something like 2001 and, though I instantly saw the appeal, I regrettably didn’t start applying it until the middle of 2005. Shortly, thereafter, I transitioned to true test-driven development.
I believe one of the most important things a test can do is validate a change to behavior. I don’t think tests are very helpful for validating changes to design, but I think they are incredibly useful for verifying that code contains the correct set of behaviors. When you’re changing behavior, it is crucial that you only modify the functionality you intended to change. That’s where a comprehensive panel of tests comes into play.
A critical ingredient of that validation is time to feedback. If it takes a day to get feedback, you might as well not have tests. If it takes an hour, you’re pushing the envelope. If it takes five minutes, it’s inconvenient, but objectively useful. Feedback needs to be so fast that you don’t feel the need to click away and do something else while you wait for it to really enable a smooth, high-performing workflow.
Many agree with me, but I’ve noticed that people focus on test-execution time.
That’s an unfortunate choice of target, as it often leads down the self-defeating path of arbitrary division so internals can be tested. This is problematic because it corrodes the quality and efficacy of your tests: The job of a test is to specify behavior, so it should only be coupled to the natural surface area of code.
It isn’t important that tests run fast. It’s important that the right tests deliver the right feedback at the right time. That is: the important thing is that you get feedback on your changes in time to do something about it and that it doesn’t disrupt your flow as a developer. Yet execution time is not the only dial we can tune in this regard.
Another solution is to set up a system where you know you only need to run some tests. For example, if you divide code into two different binary packages that you know have no direct or circuitous dependencies on one another, you don’t need to run the tests for one when you change the other.
I’ve personally had great results with this approach. I remember one product I worked on had a grossly incomplete test suite that took something like ten minutes to run. A real “test pass” was manual and took long enough to run that the team went into a six week code freeze before each release.
When asked to build a new feature for that team – something basically completely isolated from the rest of the product – I put it into its own assembly. I built a separate little update path for the assembly so it could be updated independently of the rest of the system (which was released as a monolith). Then I used TDD to develop the behavior.
The organization quickly discovered that the point of coupling I’d chosen was stable and exempted that feature from the code freeze. Which is hilarious because it was functionally complete long before the freeze started. So this one little part of the system was completely specified in tests and tuned to the point of having meaningful online help, hotkeys, and an ergonomic tab order.
All because its behavior could be completely validated in something like 2-3 seconds.
While I’m a big believer in design being the main driver for all things, including the length of your feedback cycles, it’s not the only option. Another option is updated tooling. Visual Studio has a proactive test running feature that I don’t particularly like, but will at least save you from having to manually run your tests. ReSharper has something like that, too.
Or, if you want to go a step further, you can use some more advanced, purpose built tooling. Tools like NCrunch (.NET) and Testowl (Java) solve the problem a different way: using inspection and measurement to decide which tests cover which lines of code & prioritizing most important ones.
Whatever strategy you choose – design, tooling, or something else – tests are critical, but you need to make sure you don’t have to “boil the ocean” with test-execution.