How Can I Maximize Variation Without Increasing The Number Of My Tests?

If your plan includes a long lists of Values, you are in danger of generating many more tests than you might need. The Value Expansions feature can dramatically reduce the number of tests generated in these situations.

To demonstrate how this works, let’s consider several possible test design approaches for a automobile rental application. If there are 45 vehicle models spread across 9 vehicle categories, how many values should you enter into DesignWise? Let’s examine three different approaches to this test design decision.

Strategy #1

First, let’s consider what happens if we enter all 45 vehicle models as individual values.

If we use 45 different specific car types as our Values in this example, we would need 450 tests to achieve pairwise coverage. That’s a lot of tests!

We want to create a smaller set of tests that will take less time to execute. So let’s try an alternate approach to identifying the Values that we enter into DesignWise…

Strategy #2

Let’s redefine our Values and see what happens when we use only 9 Values to describe the different categories of cars:

Hey, cool! With this second approach using 9 general car categories as our Values, we would need only 90 tests to achieve pairwise coverage.

Executing 360 fewer tests is great, but do you see a possible problem here?
One problem with this second approach is that it could lead a tester to introduce wasteful repetition into those 90 tests. Every single time the tester was asked to select a luxury car, for example, the tester might select a Porsche 911.

Strategy #3: Value Expansions

Using the Value Expansion feature will eliminate that risk. Instead of taking the risk that a tester might select Porsche 911 every single time a luxury car is specified in a test (and thereby repeat himself again and again), the tester executing tests will be prompted to execute a new and different model of luxury car which will be more effective at uncovering defects than testing the same thing repeatedly. By maximizing variation and minimizing repetition, the Value Expansions feature helps testers cover as much as possible in as few test cases as possible.

Utilizing the Value Expansions feature

On the “Inputs” screen, edit a Parameter Value to instead appear as a general category. Hover over this Value, and click on the branching icon to the Value’s right.

Edit the Expanded Values, and click on “Create” to save them.

If you would like each of these 5 “sub-values” to appear in your test scripts with equal weighting (e.g., have ‘BMW 5 Series’ appear in approximately as many test cases as ‘Porsche 911 Turbo’ would appear in), the “Use evenly” option is already set by default.

If you would rather have the sub-values that you put towards the top of your list appear more frequently in your tests than the sub-values towards the end of your list, select “Bias initial values” option.

Using the Value Expansions feature has powerful advantages over the other options. Compared to using 45 individual Values, this approach has 360 fewer tests to execute. And compared to using the 9 categories of vehicles as your Values, these test cases generated with the Value Expansions feature are: (a) more specific, (b) more varied, and (c) achieve more coverage.

A Caveat on Value Expansions

Expansions are a great way to get additional variation in your tests without sacrificing the number of tests.

But not all value expansions are guaranteed to be included in your tests.

For example, consider 10 value expansions that generate a set of tests that has 20 tests. The value will only show up a few times of that 20, maybe 5 times. Of those 5 times, only half of the value expansions will be covered. If you ‘bias’ the expanded values, even fewer still.