How to Implement N-way Constraints

Despite no explicit option currently, there are at least 4 methods that could help you successfully handle N-way relationships. For example, you might want to prevent a specific 3-way combination from appearing in your generated tests because it would be “impossible to test for”.

Method 1: Manually fixing individual tests after test generation (either with “Freezing” or post export)

Starting with the most obvious approach, you can (1) model constraints as well as you are able to do so with the constraint handling capabilities available today, (2), generate your optimized scenarios (which would erroneously include a few 3-way / n-way combinations you would like to prevent), then (3) you would manually fix the wrong combinations after they appear.

You could use DesignWise’s Freeze Feature to turn your generated set of scenarios into forced combinations of inputs. With your tests in the “frozen” state, you could then manually correct any prohibited combinations by manually over-writing any “impossible to test for scenarios” that were generated. If you expect to be modifying this set of tests multiple times, this option is recommended (because the “fixes” you make on the Frozen Tests (Forced Interactions) screen will remain in place for all future test sets you generate from this model.

Alternatively, if you will only be using the tests one time, you could simply export your generated tests and then manually overwrite the combinations you do not want to appear (in, say, Excel).

Method 2: Nested values

To demonstrate other methods, we will use a simple example where the behavior of the third parameter depends on the value combination from the first two. Specifically, if the answers to Question 1 and Question 2 are both “Yes”, then Question 3 should allow users to give a Yes or No answer. In any other combination of answers for the first two questions, the third question should not appear at all.

The idea is to represent the combinations of parameters as values.
This allows you to “flatten” the hierarchy down to 2 levels and use the Constraints as you normally do (because constraints between 2 values can be easily handled).

The method is fairly intuitive, but it has scalability limitations: with two large parameters, listing all pairs of values would be tedious and would negatively impact your flexibility in setting the algorithm strength in Scenarios. And creating an overly-long list of values for your “flattened” parameter could result in significantly increasing the number of tests generated. One mitigation strategy could be Value Expansions for pairs that behave similarly from the perspective of the N-way relationship.

Method 3: Fake Values

To understand this approach, keep in mind that DesignWise treats information as text without over-analyzing the underlying intention.

That means, as long as the string is different, it can have separate constraints despite representing the exact same variation idea.

In Q2, “Yes” is reserved for Q1 = “Yes” and “Yes*” is reserved for Q1 = “No”.

Side note: for automation (where the literal value is more important), the value expansion can be added to “Yes*” to replace the value name on export since DesignWise Automate only reads value expansions, if they exist.

Method 4: Constraints vs Forced Interactions

The last approach leverages the fact that constraints can overwrite forced interactions, and vice versa.

We start with the “base” initial setup.

Then say that Q2=Yes can never be paired with Q1=No.

Lastly, on the Forced Interactions screen, we force the combination of Q2=Yes with Q1=No and accept the warning that informs us that our constraints logic would make it impossible for that combination to ever appear. Doing so will result in the Forced Interaction over-writing the plan’s constraints logic.

The net result of this is that we would get one scenario (and only one) in our plan that would include both Q1 = No and Q2 = Yes. And the Correct Question 3 value would appear (“Not displayed”) because we forced that specific value for Question 3 to appear in the Forced Interactions screen. The other values that appear for Question 3 are properly limited from the rules on the Constraints screen.

We still get the coverage of all pairs and eliminate invalid combinations.
The downside is that the excluded pair would only be covered as many times as it would be explicitly forced, so the same scalability risk applies in situations that involve parameters with relatively large numbers of values.