Concept of report levels
Report levels are a powerful concept for controlling the level of itemization of reports. Managing large and complex test solutions with numerous test cases and possibly thousands of recording and code modules will ask for sophisticated methods to keep test reports as large and detailed as necessary, but as short as possible. The concept of report levels implements this control and is introduced herein.
In this chapter
You can download a sample solution and use it to gain practical experience for the herein introduced concepts.
- Unzip the project directory to any folder on your computer
- Start Ranorex Studio and open the solution file
Definition of report levels
We distinguish 6 pre-defined report levels with corresponding integer values.
Predefined Ranorex report levels
Important to know:
- The integer values represent an ‘attention level’, i.e. a ‘Debug’ information raises less attention than an ‘Info’ information. ‘Success’ and ‘Failure’ are the report levels with the highest attention level.
- When working with Ranorex Recorder, the herein introduced report levels are fixed.
- When working with code modules, user-defined report levels are applicable.
The concept of report levels is easily explained. It is an interplay between raising attention and being recognized. Let’s start with a simple example. Executed actions, successful, or failed test validations and inserted user-defined log messages raise report attention during a test run by attaching dedicated report levels to their status information.
Actions raising report attention by report levels
An action attaches the status message with the report level ‘Info’ (=value 20). Actions #1-#4 and actions #6-#7 are actions like this.
The recognition of report messages is controlled by test suite structuring elements. The test suite, test cases, and smart folder set the level of recognition to control which status messages are to be included in the current report.
The recognition level acts as a visibility barrier – every status message with a report level similar to the recognition level or above – is included in the current report.
Recognition level Info (value = 20)
Set recognition level = Info (value=20)
Recognition level Success (value=110)
Set recognition level = Success (value=110)
Recognition level set to Success (value=110)
Test report where only the successful test validation with status Success (value=110) is included in the report
Setting recognition level
The recognition level as the entry barrier for status messages into reports is controlled in all test suite structuring elements. Test suites, test cases, and smart folders define their report recognition level in a top-down way.
Setting report level for test suite structuring elements
Report level ‘Inherit’ means that the report level of the current structuring item is inherited from the parent structuring element. This selection enables to lower and raise the report level beginning from one structuring item down the tree-like structure in a simultaneous way.
Finally, an application example for inserting a log message and controlling the recognition level of the report is being introduced and explained. This may help you to understand the application of the concept of report levels.
Looking at the current test example we see 7 recorded actions with one validation action. Assume that:
- The test case runs stable and is robust
- The recognition level of the test case is therefore set to ‘Success’
- Only the result of the validation action is being reported
- Assume further, that, in any case, we want to be informed as soon as the ‘Submit’ button is clicked.
- This action execution shall be included in the report, while all other actions shall still be excluded if executed successfully.
Example log message
The report levels for this recording and the recognition level of the test case now look as follows.
Test report of application example
Log message with report level ‘Success’ in test report
Successful test validation with report level ‘Success’ in test report