Test Automation Best Practice #8: Write Testable Requirements

Principles of Testable Requirements

A testable requirement describes a single function or behavior of an application in a way that makes it possible to develop tests to determine whether the requirement has been met. To be testable, a requirement must be clear, measurable, and complete, without any ambiguity.

For example, assume that you are planning to test a web shopping application. You are presented with the following requirement: “Easy-to-use search for available inventory.” Testing this requirement as written requires assumptions about what is meant by ambiguous terms such as “easy-to-use” and “available inventory.” To make requirements more testable, clarify ambiguous wording such as “fast,” “intuitive” or “user-friendly.”

Requirements shouldn’t contain implementation details such as “the search box will be located in the top right corner of the screen,” but otherwise should be measurable and complete.  Consider the following example for a web shopping platform:

“When at least one matching item is found, display up to 20 matching inventory items, in a grid or list and using the sort order according to the user preference settings.”

This requirement provides details that lead to the creation of tests for boundary cases, such as no matching items, 1 or 2 matching items, and 19, 20 and 21 matching items. However, this requirement describes more than one function. It would be better practice to separate it into three separate requirements, as shown below:

• When at least one matching item is found, display up to 20 matching inventory items
• Display search results in a grid or list according to the user preference settings
• Display search results in the sort order according to the user preference settings

The principle of one function per requirement increases agility. In theory, it would be possible to release the search function itself in one sprint, with the addition of the ability to choose a grid/list display or a sort order in subsequent sprints.

Testable requirements should not include the following:

• Text that is irrelevant. Just as you can’t judge a book by the number of words, length by itself is not a sign of a testable requirement. Remove anything that doesn’t add to your understanding of the requirement.
• A description of the problem rather than the function that solves it.
• Implementation details. For implementation details such as font size, color, and placement, consider creating a set of standards that apply to the entire project rather than repeating the standards in each individual requirement.
• Ambiguity. Specifications should be specific. Avoid subjective terms that can’t be measured, such as “usually.” Replace these with objective, measurable terms such as “80%.”

Approaches to Requirements

There are a variety of approaches to writing requirements: from traditional requirements documents to more agile approaches such as user stories, test-driven development (TDD), acceptance test-driven development (ATDD), and behavior-driven development(BDD). These approaches all benefit from following principles for testable requirements.

Tools for Testable Requirements

No special tools are necessary to create testable requirements. They can be documented in word processing files or even note cards. But tools can make the process more efficient. ATDD tests may be captured and automated using a tool such as FitNesse. For BDD there are also a variety of language-specific tools to write requirements in the Gherkin GIVEN-WHEN-THEN syntax and prepare them for automation, including the following:

• Cucumber for Ruby
• Jbehave for Java
• SpecFlow for C#
• Jasmine for JavaScript

Ranorex Studio’s robust tools and open API support all testing approaches, including TDD, ATTD, and BDD. To see how Ranorex Studio integrates with SpecFlow to automate BDD scenarios, read the article How to Use Ranorex Studio in Your BDD Process on the Ranorex blog.

Conclusion

Dozens of books have been published on the topic of writing effective software requirements, and this article presents just a brief overview of strategies for ensuring that your requirements are testable. But the most important strategy is to ensure that testers and user representatives are included early in the process of requirements definition. While testable requirements make it easier to automate your tests, the key goal is to ensure that the entire team shares a clear understanding of the requirements.