Execute a test suite
In this chapter, you’ll learn about the various options to control test runs and execute tests in the test suite view.
Download the sample solution
The explanations in this chapter are based on a sample solution. You can download it below.
Unzip to any folder on your computer.
Start Ranorex Studio and open the solution file
Manage run configurations
You can include and exclude test cases and smart folders from a test run. To do so, simply check or uncheck them in the test suite.
Checkboxes to include/exclude from
Manage run configurations
Adding a new run configuration
- The run configuration can be selected from the drop-down menu.
Different run configurations
Disable/enable test suite items
Similar to including/excluding test cases and smart folders, you can also enable and disable recording modules, code modules, module groups, and
Disabling a recording module
- Disabling a module group disables all modules in this module group.
- Disabling a setup/teardown region disables all items in it.
Configure run iterations
By default, test cases and smart folders are run once during test execution. However, you may want to run them several times. You can do so with run iterations.
Configuring run iterations
The Properties pad appears to the right of the test suite.
Next to Iteration count, set the desired number of iterations.
- The number of iterations appears next to the test suite item.
- In our example, the test case is iterated 5 times.
In UI testing, errors will sometimes occur simply because the application under test was unresponsive. In these cases, one solution is to simply re-run parts of the test. You can do so with auto-retry. Test cases or smart folders with an auto-retry count will be rerun until they are either successful or all retries have been used up.
Press F4. The Properties pad appears to the right of the test suite.
Next to Retry count, set the desired number of retries.
Only test cases or smart folders that failed every single retry will be marked as failed in the report.
Configure report levels
You can also set the report level of test cases and smart folders in the test suite. Report levels allow you to control what information appears in the report, and where it appears. This is especially useful for complex tests with many test cases and smart folders, to keep the report structured.
Configure error behavior
Errors are part of testing. This is why it’s important to tell Ranorex what to do when an error occurs. You can do so by configuring the error behavior of test cases and smart folders in the test suite.
Specification of error behavior
Continue with iteration
Error behavior: Continue with iteration
The test run will continue with the next iteration of the smart folder Validation.
Continue with sibling
Error behavior: Continue with sibling
The test run will continue with the next sibling test case or smart folder. In our case, this is the smart folder DatabaseCleanUp.
Continue with parent
Error behavior: Continue with parent
The test run will continue with the next parent test case or smart folder. In our case, this is the test case CompleteDatabaseTest.
When the Stop error behavior is selected, an error immediately stops the entire test run.
Error behavior of setup/teardown regions
Setup and teardown regions follow a special, fixed error behavior.
- An error in the setup region immediately stops the test.
- If a module fails in the teardown region, the next module is run.
Run a test from the test suite view
Running a test
- Once the test
runfinishes, the report appears.
Ranorex Test Suite Runner
The Test Suite Runner is a stand-alone program which executes test suites without Ranorex Studio. It
You can use the Ranorex Test Suite Runner to execute entire test suites, run certain test cases and smart folders, or just run a specific module.
In addition, you can create new run
You can’t make changes to the test suite itself, however.
In Windows, double-click a test suite file. The file opens in the Test Suite Runner.
Stand-alone Test Suite Runner
Currently loaded test suite