TDD Is Great, Just Not for Experimental Programming

Test-driven development, or TDD, can be wonderful for elevating the quality of production software. Plenty of important software in the world, though, falls outside that category. What does TDD offer experimental programming? Not so much. Here’s why.

Spreadsheets rule

TDD is a software programming practice that emphasizes a short development cycle for incremental construction of high-quality, maintainable applications and libraries. TDD yields defect reduction, requirements traceability, code quality, comprehensive regression tests, and other benefits — in some cases.

However, there are times not to use TDD, or to otherwise restrict its application. Considerable (if not entirely conclusive) evidence posits that inspection is even better than TDD in several regards.

Nearly all the discussion around that position is focused on commercial-like programming, and often it emphasizes what I call algorithmic programming. This is almost certainly a minority of all programming. “Spreadsheets are programs and Excel is the most popular software development environment in the world!” Hjalmar Gislason rightly proclaims.

Even in the world of more conventional programs, the emphasis is increasingly on integration rather than “hand-coded” algorithms.

Consider experimental programming

One theme that unifies many of these outliers of programming is that they are experimental or exploratory: Rather than a program that will give the right answer when run millions of times, the point is to program just enough to decide on an action outside the programming world.

Consider these examples:

• A family tosses together a crude spreadsheet calculation to determine whether refinancing a home mortgage might be beneficial.
• A full-time commercial programmer who practices TDD ties together several licensed APIs to get a feel for the feasibility of outsourcing a thousand requests a second of new functionality to external services.
• A scientist writes a couple of hundred lines of Java or R or Python to determine whether certain observations are worth exploring as researchable signal or seem instead to be just random noise.
• A security investigator automates queries against log files in pursuit of likely malicious invaders.
• A DevOps specialist quickly scripts enough bespoke measurements to diagnose a thorny problem with testing in production.

Replicability, correctness, and other traditional programming metrics are good to have in all these cases. They’re not top priorities, though. Understanding that informs follow-up actions is a higher goal, for instance.

Good TDD might be an advantage in these situations, but it’s not a requirement. Flexibility — the ability to explore possibilities that weren’t even considered when the experiment began — is more essential. Rapid results are also more important.

One complement to TDD in experimental programming is coverage tooling. While most of the examples above are so lightweight that they don’t typically consider coverage, modern coverage tools help ensure that even experimental results are worth trusting.

Context is everything

At its best, TDD is rewarding and even life-altering. However, TDD poorly fits many other common programming situations.

It’s important to be sensitive to context. If your programming emphasizes integration of external dependencies, one-time-only results, answers that you won’t be able to recognize ahead of time, or techniques that blur the distinction between code and data, then you need to plan to use techniques other than TDD to keep your quality high.