Ranorex standard reporting | Ranorex
Help CenterUser GuideRanorex Studio fundamentals

Ranorex standard reporting

Ranorex standard reporting describes all reporting issues performed from within Ranorex Recorder and applying the pre-defined Ranorex reports. The purpose of this chapter is to introduce you the naming conventions of reports and their storage location as well as the report components and how to read and interpret Ranorex standard reports. Finally, it shows you how to navigate through the report and how to use and apply the filter and search functions.

In this chapter

    Report name & storage location

    This section is to introduce and explain naming conventions and storage locations of Ranorex standard reports.

    Test report naming

    Every test report is given an automatically generated name. The naming conventions are introduced and explained herein.

    Automated report name

    Automated report name

    Important to know:

    • A test suite run generates a test report with the test suite name at the beginning
    • A recording test run generates a test report with the recording module name at the beginning
    • The second part of the report name is a combination of the date and time where the report is created:
      • Test date = ‘February, 21 2018’ = ‘2018221
      • Test time = ‘7:08:36 AM’ = ‘070836
    • The combination is done by means of ‘_
    • The file ending is ‘rxlog” which is an abbreviation of Ranorex Log

    Further reading

    Report naming conventions can be configured. Find the corresponding introduction and explanation in > Ranorex Studio system details > Settings & configuration > ⇢ Report settings.

    Test reports in project file view

    Test reports are stored in a folder /Reports/ within the project file view of Ranorex Studio.

    Test report storage location

    Test report storage location

    Important to know:

    • Every test run generates a test report if reporting is activated in the settings
    • Every test report is stored in the before mentioned folder
    • Report files can be opened without running Ranorex Studio by directly accessing the file

    Further reading

    The options for disabling reporting and how to change the report viewer options are introduced and explained in > Ranorex Studio system details > Settings & configuration > ⇢ Report settings.

    Report files in project file structure

    Report files in default storage location

    Report files in default storage location

    /Reports/ folder in project file structure
    Report data file and report file of first test run
    Report data file and report file of second test run

    Basic report components

    This section outlines the basic components of reports which are printed at the beginning of each report.

    Basic report data

    Basic report data

    Test suite name / recording module name

    The headline contains the name of the corresponding test container, i.e. the test suite, or the recording module which was the source of the test run.

    Test information, containing:

    • computer name, or endpoint
    • operating system (‘Windows 10 64bit’)
    • OS language information (‘US-english’)
    • date and time of test execution (‘February 21, 2018; 7:08:36 AM’)
    • screen resolution (‘5,760 x 1,200 pixels’)
    • test run duration (‘6.25 seconds’)
    Error/warning counter

    • The last section of the information panel of a test report contains a counter for errors and warnings
    • If at least, one warning occurs within the test report, this is visibly indicated below the warning counter
    Error/warning counter in test report

    Error/warning counter in test report

    Test result summary

    The test case result summary is primarily represented by a pie chart. The color of the pie chart indicates if a test result was successful, or not.

    Types of test report summary

    Types of test report summary

    Successful test summary

    A green pie chart basically indicates a successful test run

    Failed test summary

    A red pie chart indicates a failed test run

    Test summary

    • Pie chart with successful failed and blocked test cases
    • It does not contain results from smart folders. Smart folder details can be retrieved from the test report itself
    • See an example where 8 test cases were planned to be executed within a test run. Two of them have been passed successfully, one was failed – the test run was aborted, and the remaining 5 test cases have been blocked

    Expanding/collapsing report details

    Test report details can be expanded, or collapsed by clicking on the opening/closing symbols prior to the name of the corresponding module.

    Expanding/collapsing report details

    Expanding/collapsing report details

    • Click into the symbols to show/hide report details
    • Test modules are nested within the test report
    tipp icon

    Hint

    The test report structure of nested test modules follows the test suite structure.

    Detailed report contents

    The report contains detailed activity messages of the test run.

    Detailed report content structure

    Detailed report content structure

    Time

    Each message line begins with the relative execution time measured from the beginning of the test run. This can be changed within the settings.

    Report level

    The second information represents the ‘report level’ of the message. This level has been explained in the previous chapter.

    Report category

    The ‘Category’ indicates the action type performed.

    Message contents

    The message itself contains what happened during the execution of the action in the test run (see below example image).

    Example for a report message content

    Example for a report message content

    Important to know:

    • The report provides a screenshot at the failure level automatically (if tracing screenshots is enabled)
    • To support debugging, a screenshots will also be provided for the action prior to the failure
    Screenshots in report

    Screenshots in report

    Screenshot for action prior to failed test action

    Screenshot for failed test action

    Simple report structure

    The structure of the report basically follows the test suite structure and the sequence of executed actions. Each test suite structuring element is present within the report and its contents are visible / partly visible/invisible due to the settings of the report level.

    Simple report structure

    Simple report structure

    Test report structure representing corresponding test suite structure
    Report messages representing action list of corresponding recording module

    Data-iterations in reports

    If test cases and smart folders are bound to a set of test data and are executed in an iterated way with a changing set of data, these data iterations find their representation within the test report, too.

    Data iterations in reports

    Data iterations in reports

    Indication of data iterations

    • Data iterations are indicated by the “Rows: #” and “Data Row: #” labels next to test containers.
    • “Rows #” indicates how many data rows the data container will be iterated.
    Representation of data rows in test report

    • Each data row finds its representation within the structured test report
    Variable and value

    • Variable and value of each iterated data item

    Important to know:

    • Secret test data can be masked (i.e. made invisible to readers) by setting the corresponding configuration
    • If, for example, age and gender of a person are to be ‘masked’, this will lead to a test report as shown below
    Masked report data

    Masked report data

    Reference

    Data iterations, masking data in reports are part of the advanced topic > Ranorex Studio advanced > ⇢ Data-driven testing.

    Run iterations in test reports

    If a test case or a smart folder is run iteratively, the run iterations are displayed in the test report as well.

    Run iterations in test reports

    Run iterations in test reports

    Important to know:

    • Run iterations are indicated by the “Iterations: #” and “Run #” labels next to test containers
    • “Iterations: #” tells you how many times the test container will be run

    Reference

    Run iterations are part of the fundamental topic > Ranorex Studio fundamentals > Test suite > ⇢ Running tests.

    Filtering messages

    Use the checkboxes of the filter area in the upper part of the test report to filter the log messages.

    Filtering report messages

    Filtering report messages

    Success – displays report messages with report level ‘Success’ if checked – and – if these messages are included in the report based on report levels. Hides these messages if not checked.
    Failed – displays report messages with report level ‘Failure’. Hides these messages if not checked.
    Blocked – displays blocked test cases of a test run when checked. Hides blocked test cases if not checked.
    Note icon

    Note

    Blocked test cases or smart folders are test units which have not been executed during a test run due to a test abortion, or which are not selected for run execution within the test suite.

    Jump to item

    You can directly jump from the report message to the corresponding action item/test case/smart folder in Ranorex Studio.

    Click Expand details in test report
    Move the mouse to any report line of interest
    See the appearing button Jump to item at the right side of the report
    Jump to item functionality

    Jump to item functionality

    Jump to item button
    Corresponding jump source with linked repository item

    Open in Spy

    If you open a stand-alone test report from outside of Ranorex Studio, the Jump to item button is supplemented by an Open in Spy direct link. Click the button to open the corresponding report item in Ranorex Spy.

    Open in Spy functionality

    Open in Spy functionality

    Open in Spy report functionality for stand-alone reports

    Progressive report preview

    Ranorex allows catching the progress of the report of a long test run. So, you do not have to wait for the final report at the end of the test but follow its progress. This section shows how to do it.

    Important to know:

    • The creation of any report is started with the test run start
    • The initial storage happens at the end of the ‘auto-save’ time duration (default = 30s)
    • Test reports can be opened outside of Ranorex by simply double-clicking the report file
    Progressive reporting

    Progressive reporting

    Result(s):

    • During a longer test run, two button indicators tell you that the report is still in progress
    tipp icon

    Hint

    • Avoid trying to open the report from the machine the current test runs.
    • This will lead to a test abortion due to the fact, that you need to use the mouse while the current test also uses your mouse for UI-interaction

    Report settings & configuration

    The storage location for reports, the previously introduced naming conventions, the auto-save time duration, and all other report-specific configurations can be set within the settings dialog of Ranorex Studio.

    Report settings

    Report settings

    Select the test suite and open the context menu
    Click Properties
    In the settings window, click the Report register tab

    Further reading

    A full overview of all report settings & configurations are introduced and explained in > Ranorex Studio system details > Settings & configuration > ⇢ Report settings.