Report levels categorize events that happen during a test run and govern what information is included in the report. When your tests contain hundreds of test cases with thousands of modules, report levels are the key to striking the balance between detail and brevity. In this chapter, you’ll find out how report levels work and how to use them.
In this chapter
Download the sample solution
The explanations in this chapter are based on a sample solution that you can download below.
Install the sample solution:
Unzip to any folder on your computer.
Start Ranorex Studio and open the solution file
What report levels are there?
Ranorex Studio comes with 6 default report levels, each with a corresponding color and integer value.
Default Ranorex report levels
- The higher the integer value of a report level, the higher its attention level, i.e. a Debug notification raises less attention than an Info notification.
- In recording modules, the default report levels apply.
- In code modules, you can define your own report levels.
The concept of report levels is easily explained. It is an interplay between raising attention and being recognized. Let’s start with a simple example. Executed actions, successful, or failed test validations and inserted user-defined log messages raise report attention during a test run by attaching dedicated report levels to their status information.
Actions raising report attention by report levels
Actions #1-#4 and #6-#7 have the report level Info (value = 20).
A validation virtually always either succeeds or fails. Therefore, a successful test validation (action #5) has the report level Success (value = 110). If it had failed, it would have the report level Failure (value = 120).
The recognition of report messages is controlled by test suite structuring elements. The test suite, test cases, and smart folder set the level of recognition to control which status messages are to be included in the current report.
Recognition level Info (value = 20)
Recognition level set to Info (value = 20)
Recognition level Success (value=110)
Recognition level set to Success (value = 110)
Test report where only the successful test validation with status Success (value = 110) is included in the report
Setting recognition level
The recognition level as the entry barrier for status messages into reports is controlled in all test suite structuring elements. Test suites, test cases, and smart folders define their report recognition level in a top-down way.
Change to the test suite view.
Right-click the test suite, a test case, or a smart folder.
Click Report level and select the desired value.
When this option is selected, the report level is inherited from the direct-parent structuring item.
Let’s apply the above explanations to an example. We’ll insert a log message and set its report level.
The recording module in the sample solution contains 7 recorded actions, one of them a validation action.
The recognition level of the test case is set to Success. This means that currently, only the successful or failed validation would be reported during a test run.
We now want to add a log message which tells us that the Submit button has been clicked, and of course we want it to show up in the report, whereas all other actions except for the validation shouldn’t show up.
Add a log message after action #4.
The report levels for the actions in this recording and the recognition level of the test case now look like this:
Test report of application example
Log message with report level Success in report.
Successful test validation with report level Success in report.