Testing is a crucial part of the software creation process. It ensures that your code is working correctly and that all bugs are found before the software lands in the hands of consumers. But it can be difficult to explain these tests and their results to investors...
How to Implement N-way Constraints
Despite no explicit option currently, there are at least 4 methods that could help you successfully handle N-way relationships. For example, you might want to prevent a specific 3-way combination from appearing in your generated tests because it would be “impossible to test for”.
Method 1: Manually fixing individual tests after test generation (either with “Freezing” or post export)
Starting with the most obvious approach, you can (1) model constraints as well as you are able to do so with the constraint handling capabilities available today, (2), generate your optimized scenarios (which would erroneously include a few 3-way / n-way combinations you would like to prevent), then (3) you would manually fix the wrong combinations after they appear.
You could use DesignWise’s Freeze Feature to turn your generated set of scenarios into forced combinations of inputs. With your tests in the “frozen” state, you could then manually correct any prohibited combinations by manually over-writing any “impossible to test for scenarios” that were generated. If you expect to be modifying this set of tests multiple times, this option is recommended (because the “fixes” you make on the Frozen Tests (Forced Interactions) screen will remain in place for all future test sets you generate from this model.
Alternatively, if you will only be using the tests one time, you could simply export your generated tests and then manually overwrite the combinations you do not want to appear (in, say, Excel).
Method 2: Nested values
To demonstrate other methods, we will use a simple example where the behavior of the third parameter depends on the value combination from the first two. Specifically, if the answers to Question 1 and Question 2 are both “Yes”, then Question 3 should allow users to give a Yes or No answer. In any other combination of answers for the first two questions, the third question should not appear at all.
The method is fairly intuitive, but it has scalability limitations: with two large parameters, listing all pairs of values would be tedious and would negatively impact your flexibility in setting the algorithm strength in Scenarios. And creating an overly-long list of values for your “flattened” parameter could result in significantly increasing the number of tests generated. One mitigation strategy could be Value Expansions for pairs that behave similarly from the perspective of the N-way relationship.
Method 3: Fake Values
To understand this approach, keep in mind that DesignWise treats information as text without over-analyzing the underlying intention.
That means, as long as the string is different, it can have separate constraints despite representing the exact same variation idea.
In Q2, “Yes” is reserved for Q1 = “Yes” and “Yes*” is reserved for Q1 = “No”.
Side note: for automation (where the literal value is more important), the value expansion can be added to “Yes*” to replace the value name on export since DesignWise Automate only reads value expansions, if they exist.
Method 4: Constraints vs Forced Interactions
The last approach leverages the fact that constraints can overwrite forced interactions, and vice versa.
We start with the “base” initial setup.
Then say that Q2=Yes can never be paired with Q1=No.
The net result of this is that we would get one scenario (and only one) in our model that would include both Q1 = No and Q2 = Yes. And the Correct Question 3 value would appear (“Not displayed”) because we forced that specific value for Question 3 to appear in the Forced Interactions screen. The other values that appear for Question 3 are properly limited from the rules on the Constraints screen.