Concept of report levels | Ranorex
Help CenterUser GuideRanorex Studio fundamentals

Report levels

Report levels categorize events that happen during a test run and govern what information is included in the report. When your tests contain hundreds of test cases with thousands of modules, report levels are the key to striking the balance between detail and brevity. In this chapter, you’ll find out how report levels work and how to use them.

In this chapter

    Download the sample solution

    The explanations in this chapter are based on a sample solution that you can download below.

    Sample solution

    Theme: Build a test
    Time: 15 minutes

    Download sample file

    Install the sample solution:

    Unzip to any folder on your computer.
    Start Ranorex Studio and open the solution file RxDatabase.rxsln

    tipp icon

    Hint

    The sample solution is available for Ranorex versions 8.0 or higher. You must agree to the automatic solution upgrade for versions 8.2 and higher.

    What report levels are there?

    Ranorex Studio comes with 6 default report levels, each with a corresponding color and integer value.

    Predefined Ranorex report levels

    Default Ranorex report levels

    • The higher the integer value of a report level, the higher its attention level, i.e. a Debug notification raises less attention than an Info notification.
    • In recording modules, the default report levels apply.
    • In code modules, you can define your own report levels.

    Raising attention

    The concept of report levels is easily explained. It is an interplay between raising attention and being recognized. Let’s start with a simple example. Executed actions, successful, or failed test validations and inserted user-defined log messages raise report attention during a test run by attaching dedicated report levels to their status information.

    Actions raising report attention by report levels

    Actions raising report attention by report levels

    Actions #1-#4 and #6-#7 have the report level Info (value = 20).

    A validation virtually always either succeeds or fails. Therefore, a successful test validation (action #5) has the report level Success (value = 110). If it had failed, it would have the report level Failure (value = 120).

    Report recognition

    The recognition of report messages is controlled by test suite structuring elements. The test suite, test cases, and smart folder set the level of recognition to control which status messages are to be included in the current report.

    tipp icon

    Hint

    The recognition level acts as a visibility barrier – every status message with a report level similar to the recognition level or above – is included in the current report.

    Recognition level Info (value = 20)

    Set recognition level = Info (value=20)

    Recognition level set to Info (value = 20)

    Test report where all status messages with report level Info and above are included

    Recognition level Success (value=110)

    Set recognition level = Success (value=110)

    Recognition level set to Success (value = 110)

    Test report where only the successful test validation with status Success (value = 110) is included in the report

    Setting recognition level

    The recognition level as the entry barrier for status messages into reports is controlled in all test suite structuring elements. Test suites, test cases, and smart folders define their report recognition level in a top-down way.

    Change to the test suite view.

    Right-click the test suite, a test case, or a smart folder.

    Click Report level and select the desired value.

    Setting report level for test suite structuring elements

    Inherit

    When this option is selected, the report level is inherited from the direct-parent structuring item.

    Application example

    Let’s apply the above explanations to an example. We’ll insert a log message and set its report level.

    Initial situation

    The recording module in the sample solution contains 7 recorded actions, one of them a validation action.

    The recognition level of the test case is set to Success. This means that currently, only the successful or failed validation would be reported during a test run.

    We now want to add a log message which tells us that the Submit button has been clicked, and of course we want it to show up in the report, whereas all other actions except for the validation shouldn’t show up.

    Here’s how:

    Add a log message after action #4.

    Example log message
    Enter a message and set its report level to Success.

    Result

    The report levels for the actions in this recording and the recognition level of the test case now look like this:

    Test report of application example

    Test report of application example

    Log message with report level Success in report.

    Successful test validation with report level Success in report.