Ranorex standard reporting

In this chapter, you’ll learn where reports are stored, what they contain, how to read them, and where you can change report settings.

In this chapter

File name and location

File name

The file name of a report is generated automatically. It changes depending on whether the test is executed from a recording module or from a test suite.

• For a run from a test suite, the file name starts with the test suite’s name.
• For a run from a recording, the file name starts with the recording’s name.
• The second part of the report name is a combination of the date and time the report is generated.
  • Test date = 21 February 2018 = 20180221
  • Test time = 07:08:36 PM = 190836
• Date and time are separated by an underscore.
• The file ending is .rxlog, short for Ranorex Log

Report file location

In the projects view, reports are stored in the Reports folder of your project.

You can also access reports in the same folder in Windows. Ranorex Studio doesn’t have to be running when you open a report this way.

/Reports/ folder in your project’s folder.

Report file and raw data file of a recording test run.

Layout files for the report.

Report header

The report header displays a range of useful data and summarizes the test results.

Test suite name/recording module name

The name of the test suite or recording module from which the test was run.

System and test data

Error/warning counters

If there’s at least one warning in the report, an orange notification appears, as shown below.

Test results

A pie chart summarizing the test results.

Successful test

A green pie chart means all test cases were successful.

Failed test

A red pie chart means all test cases failed.

Test summary

• Pie chart with successful, failed, and blocked test cases
• In the example, the run configuration contained 8 test cases. The first two test cases passed successfully. The third failed, after which the test was aborted, blocking the remaining 5 test cases.

Viewing report details

By default, the details for test cases, smart folders, etc. are collapsed. Simply click the arrow next to an item’s name to show the details.

Basic report structure

The report is structured in the same way as the test suite or recording module on which the test run was based. Each structure element in the test suite appears the report, as well as each test action item. Report messages appear for all actions in the order of execution provided that they met the report level threshold.

The Recording1 item contains all the report messages for the actions in it.

Detailed report contents

Time

By default, this column displays the relative execution time measured from the beginning of the test run. This can be changed in the ⇢ report settings.

Level

The second column shows the report level of the action performed.

Category

This column shows the action type performed.

Message

The report message contains more detailed information as to what happened during the execution of the action.

In case of a failure, the report will also include two screenshots, one at the moment of failure and one right before the failure occurred.

Screenshot immediately before the failure.

Screenshot during failure.

Data iterations in reports

In data-driven tests, a test case or smart folder bound to data goes through a number of iterations depending on the data source. In the report, each of these iterations is shown separately.

Data iterations

• Data iterations are indicated by the Rows: # labels next to test containers.
• Rows # indicates how many times the data container will be iterated – one time for each row.

Data rows in test report

• Each data row/iteration and its details are shown in the report.

Variable and value

• Variables and values used in this iteration.

Run iterations in test reports

If a test case or a smart folder is run iteratively, each run iteration appears in the report with its details.

• Run iterations are indicated by the Iterations: # and Run: # labels next to test containers.
• Iterations: # tells you how many times the test container is run.

Filter messages

There are two filters available in the report: one for filtering test containers and one for filtering messages for actions.

Filter test containers

Use the checkboxes in the upper part of the report to filter test containers by their success status.

Success: displays report messages with report level ‘Success’ if checked, and if these messages are included in the report based on report levels. Hides these messages if not checked.

Failed: displays report messages with report level ‘Failure’. Hides these messages if not checked.

Blocked: displays blocked test cases of a test run when checked. Hides blocked test cases if not checked.

Filter action messages

You can also filter action messages in modules by their report level.

Jump to item

You can directly jump from the report message to the corresponding action or test suite item in Ranorex Studio.

Mouse over a report entry, i.e. a test container or a report message for an action.

Jump to item button.

Corresponding action with linked repository item.

Open in Spy

When you open a report outside of Ranorex Studio, the Open in Spy button becomes available. Click the button to open the corresponding report item in Ranorex Spy.

Open in Spy report functionality.

Progressive report preview

Ranorex generates the report as the test run progresses. You can view it at any time during the test run. This is especially useful for very long test runs.

• Ranorex starts generating the report as soon as the test run begins.
• The report file is saved once the defined auto-save time elapses (default = 30s).
• At this point, you can open the in-progress report simply by double-clicking the report file.

Report settings and configuration

To access the report settings: