Ranorex standard reporting

Ranorex standard reporting describes all reporting issues performed from within Ranorex Recorder and applying the pre-defined Ranorex reports. The purpose of this chapter is to introduce you the naming conventions of reports and their storage location as well as the report components and how to read and interpret Ranorex standard reports. Finally, it shows you how to navigate through the report and how to use and apply the filter and search functions.

In this chapter

Report name & storage location

This section is to introduce and explain naming conventions and storage locations of Ranorex standard reports.

Test report naming

Every test report is given an automatically generated name. The naming conventions are introduced and explained herein.

Important to know:

• A test suite run generates a test report with the test suite name at the beginning
• A recording test run generates a test report with the recording module name at the beginning
• The second part of the report name is a combination of the date and time where the report is created:
  • Test date = ‘February, 21 2018’ = ‘2018221’
  • Test time = ‘7:08:36 AM’ = ‘070836’
• The combination is done by means of ‘_’
• The file ending is ‘rxlog” which is an abbreviation of Ranorex Log

Test reports in project file view

Test reports are stored in a folder /Reports/ within the project file view of Ranorex Studio.

Important to know:

• Every test run generates a test report
• Every test report is stored in the before mentioned folder
• Report files can be opened without running Ranorex Studio by directly accessing the file

Report files in project file structure

/Reports/ folder in project file structure

Report data file and report file of first test run

Report data file and report file of second test run

Basic report components

This section outlines the basic components of reports which are printed at the beginning of each report.

Test suite name / recording module name

The headline contains the name of the corresponding test container, i.e. the test suite, or the recording module which was the source of the test run.

Test information, containing:

• computer name, or endpoint
• operating system (‘Windows 10 64bit’)
• OS language information (‘US-english’)
• date and time of test execution (‘February 21, 2018; 7:08:36 AM’)
• screen resolution (‘5,760 x 1,200 pixels’)
• test run duration (‘6.25 seconds’)

Error/warning counter

• The last section of the information panel of a test report contains a counter for errors and warnings
• If at least, one warning occurs within the test report, this is visibly indicated below the warning counter

Test result summary

The test case result summary is primarily represented by a pie chart. The color of the pie chart indicates if a test result was successful, or not.

Successful test summary

A green pie chart basically indicates a successful test run

Failed test summary

A green pie chart basically indicates a successful test run

Test summary

• Pie chart with successful failed and blocked test cases
• It does not contain results from smart folders. Smart folder details can be retrieved from the test report itself
• See an example where 8 test cases were planned to be executed within a test run. Two of them have been passed successfully, one was failed – the test run was aborted, and the remaining 5 test cases have been blocked

Expanding/collapsing report details

Test report details can be expanded, or collapsed by clicking on the opening/closing symbols prior to the name of the corresponding module.

• Click into the symbols to show/hide report details
• Test modules are nested within the test report

Detailed report contents

The report contains detailed activity messages of the test run.

Time

Each message line begins with the relative execution time measured from the beginning of the test run

Report level

The second information represents the ‘report level’ of the message. This level is introduced and explained the next chapter.

Report category

The ‘Category’ indicates the action type performed.

Message contents

The message itself contains what happened during the execution of the action in the test run (see below example image).

Important to know:

• Consider that the report information at the failure level provides a screenshot automatically (if tracing screenshots is enabled).
• To visually retrace the last steps that lead to the current error, screenshots will be provided for the last three actions.

Available screenshot prior to failed test action

Available screenshot for failed test action

Simple report structure

The structure of the report basically follows the test suite structure and the sequence of executed actions. Each test suite structuring element is present within the report and its contents are visible / partly visible/invisible due to the settings of the report level.

Test report structure representing corresponding test suite structure

Report messages representing action list of corresponding recording module

Data-iterations in reports

If test cases and smart folders are bound to a set of test data and are executed in an iterated way with a changing set of data, these data iterations find their representation within the test report, too.

Indication of data iterations

• Data iterations are indicated by the “Rows: #” and “Data Row: #” labels next to test containers.
• “Rows #” indicates how many data rows the data container will be iterated.

 

Representation of data rows in test report

• Each data row finds its representation within the structured test report

 

Iterated action representation in test report

• Each iterated action finds its representation in the test report

 

Important to know:

• Secret test data can be masked (i.e. made invisible to readers) by setting the corresponding configuration
• If, for example, age and gender of a person are to be ‘masked’, this will lead to a test report as shown below

Run iterations in test reports

If a test case or a smart folder is run iteratively, the run iterations are displayed in the test report as well.

Important to know:

• Run iterations are indicated by the “Iterations: #” and “Run #” labels next to test containers
• “Iterations: #” tells you how many times the test container will be run

Filtering messages

Use the checkboxes of the filter area in the upper part of the test report to filter the log messages.

Success – displays report messages with report level ‘Success’ if checked – and – if these messages are included in the report based on report levels. Hides these messages if not checked.

Failed – displays report messages with report level ‘Failure’. Hides these messages if not checked.

Blocked – displays blocked test cases of a test run when checked. Hides blocked test cases if not checked.

Jump to item

You can directly jump from the report message to the corresponding action item/test case/smart folder in Ranorex Studio.

Click Expand details in test report

Move the mouse to any report line of interest

See the appearing button Jump to item at the right side of the report

Jump to item button

Corresponding jump source with linked repository item

Open in Spy

If you open a stand-alone test report from outside of Ranorex Studio, the Jump to item button is supplemented by an Open in Spy direct link. Click the button to open the corresponding report item in Ranorex Spy.

Open in Spy report functionality for stand-alone reports

Progressive report preview

Ranorex allows catching the progress of the report of a long test run. So, you do not have to wait for the final report at the end of the test but follow its progress. This section shows how to do it.

Important to know:

• The creation of any report is started with the test run start
• The initial storage happens at the end of the ‘auto-save’ time duration (default = 30s)
• Test reports can be opened outside of Ranorex by simply double-clicking the report file

Result(s):

• During a longer test run, two button indicators tell you that the report is still in progress

Report settings & configuration

The storage location for reports, the previously introduced naming conventions, the auto-save time duration, and all other report-specific configurations can be set within the settings dialog of Ranorex Studio.

Change to test suite view and click Properties of the context menu

Click the Report register tab of the settings dialog