Ranorex standard reporting

Ranorex standard reporting describes all reporting issues performed from within Ranorex Recorder and applying the pre-defined Ranorex reports. The purpose of this chapter is to introduce you the naming conventions of reports and their storage location as well as the report components and how to read and interpret Ranorex standard reports. Finally, it shows you how to navigate through the report and how to use and apply the filter and search functions.

In this chapter

Report name & storage location

This section is to introduce and explain naming conventions and storage locations of Ranorex standard reports.

Test report naming

Every test report is given an automatically generated name. The naming conventions are introduced and explained herein.

Important to know:

• A test suite run generates a test report with the test suite name at the beginning
• A recording test run generates a test report with the recording module name at the beginning
• The second part of the report name is a combination of the date and time where the report is created:
  • Test date = ‘February, 21 2018’ = ‘2018221’
  • Test time = ‘7:08:36 AM’ = ‘070836’
• The combination is done by means of ‘_’
• The file ending is ‘rxlog” which is an abbreviation of Ranorex Log

Test reports in project file view

Test reports are stored in a folder /Reports/ within the project file view of Ranorex Studio.

Important to know:

• Every test run generates a test report if reporting is activated in the settings
• Every test report is stored in the before mentioned folder
• Report files can be opened without running Ranorex Studio by directly accessing the file

Report files in project file structure

Basic report components

This section outlines the basic components of reports which are printed at the beginning of each report.

The headline contains the name of the corresponding test container, i.e. the test suite, or the recording module which was the source of the test run.

• computer name, or endpoint
• operating system (‘Windows 10 64bit’)
• OS language information (‘US-english’)
• date and time of test execution (‘February 21, 2018; 7:08:36 AM’)
• screen resolution (‘5,760 x 1,200 pixels’)
• test run duration (‘6.25 seconds’)

• The last section of the information panel of a test report contains a counter for errors and warnings
• If at least, one warning occurs within the test report, this is visibly indicated below the warning counter

The test case result summary is primarily represented by a pie chart. The color of the pie chart indicates if a test result was successful, or not.

A green pie chart basically indicates a successful test run

Failed test summary

A red pie chart indicates a failed test run

• Pie chart with successful failed and blocked test cases
• It does not contain results from smart folders. Smart folder details can be retrieved from the test report itself
• See an example where 8 test cases were planned to be executed within a test run. Two of them have been passed successfully, one was failed – the test run was aborted, and the remaining 5 test cases have been blocked

Expanding/collapsing report details

Test report details can be expanded, or collapsed by clicking on the opening/closing symbols prior to the name of the corresponding module.

• Click into the symbols to show/hide report details
• Test modules are nested within the test report

Detailed report contents

The report contains detailed activity messages of the test run.

Each message line begins with the relative execution time measured from the beginning of the test run. This can be changed within the settings.

Report level

The second information represents the ‘report level’ of the message. This level has been explained in the previous chapter.

The ‘Category’ indicates the action type performed.

The message itself contains what happened during the execution of the action in the test run (see below example image).

Important to know:

• The report provides a screenshot at the failure level automatically (if tracing screenshots is enabled)
• To support debugging, a screenshots will also be provided for the action prior to the failure

Screenshot for action prior to failed test action

Screenshot for failed test action

Simple report structure

The structure of the report basically follows the test suite structure and the sequence of executed actions. Each test suite structuring element is present within the report and its contents are visible / partly visible/invisible due to the settings of the report level.

Data-iterations in reports

If test cases and smart folders are bound to a set of test data and are executed in an iterated way with a changing set of data, these data iterations find their representation within the test report, too.

• Data iterations are indicated by the “Rows: #” and “Data Row: #” labels next to test containers.
• “Rows #” indicates how many data rows the data container will be iterated.

• Each data row finds its representation within the structured test report

• Variable and value of each iterated data item

Important to know:

• Secret test data can be masked (i.e. made invisible to readers) by setting the corresponding configuration
• If, for example, age and gender of a person are to be ‘masked’, this will lead to a test report as shown below

Run iterations in test reports

If a test case or a smart folder is run iteratively, the run iterations are displayed in the test report as well.

Important to know:

• Run iterations are indicated by the “Iterations: #” and “Run #” labels next to test containers
• “Iterations: #” tells you how many times the test container will be run

Filtering messages

Use the checkboxes of the filter area in the upper part of the test report to filter the log messages.

Jump to item

You can directly jump from the report message to the corresponding action item/test case/smart folder in Ranorex Studio.

Open in Spy

If you open a stand-alone test report from outside of Ranorex Studio, the Jump to item button is supplemented by an Open in Spy direct link. Click the button to open the corresponding report item in Ranorex Spy.

Progressive report preview

Ranorex allows catching the progress of the report of a long test run. So, you do not have to wait for the final report at the end of the test but follow its progress. This section shows how to do it.

Important to know:

• The creation of any report is started with the test run start
• The initial storage happens at the end of the ‘auto-save’ time duration (default = 30s)
• Test reports can be opened outside of Ranorex by simply double-clicking the report file

Result(s):

• During a longer test run, two button indicators tell you that the report is still in progress

Report settings & configuration

The storage location for reports, the previously introduced naming conventions, the auto-save time duration, and all other report-specific configurations can be set within the settings dialog of Ranorex Studio.