Basic report characteristics & data
Independent from the use and application of Ranorex reports, there are report characteristics and report data which are shared across all types of reports. The purpose of this chapter is to introduce and explain them.
In this chapter
Reporting starts with the beginning of the test run. Find a summary of the basic report characteristics below:
- Data collection is started synchronously by starting a test run.
- Type and amount of report data which is to be included in a current report are controlled by report levels, a concept being introduced in the next chapter.
- Test solutions may contain numerous test cases with dozens/hundreds of structuring elements and test items. Therefore, test runs may last from minutes to hours. Report data usually is available in a progressive way right from the beginning of the test run. This principle is also explained later in this chapter.
Reported action execution
Basically, every action which is executed during a test run triggers a status message by default and therefore can be included in a test report.
Example for reported action execution
Important to know:
- Each action execution may be present as one report item in the report
- Test cases and test runs usually are large and can contain thousands of actions to be executed. The representation of all action executions within the report, therefore, will lead to large and unmanageable reports
- The concept of error-levels helps control which test run events find their way into the report
- Furthermore, the report access of an action can be controlled by its action properties
Controlling the report access of actions
Controlling report access of actions
The default setting for logging the action item may be ‘true’, or ‘false’. This is specified within the settings of Ranorex Studio.
The complete options for setting and configuring reports are introduced and explained in > Ranorex Studio system details > Settings & configuration > ⇢ Report settings.
Reported test validation
Test validation is a special type of action which demands special interest with respect to reporting.
Successful & failure test validation in report
A successfully executed test validation action attaches its status message with the report level ‘Success’ and is usually printed in green.
A failed validation action usually leads to the abortion of the test run and attaches its status message with the report level ‘Failure’ and is printed in red.
The default abortion of a test run after a failed test validation might not be useful. Therefore, it can be ‘enabled to continue on fail’.
Enabling an action to continue on failure is introduced along with the management of actions in > Ranorex Studio fundamentals > Actions > ⇢ Managing actions.
Controlling the report access of test validation
The access of test validations to reports can be controlled through the property pane of validation actions.
Report access on success & failure for test validation
Recording separators allow structuring recording modules and provide an easy way for making a test run more transparent through report messages.
Separators are also part of the various options for managing recordings. Therefore, it is also introduced and explained in >
Ranorex Studio fundamentals > Ranorex Recorder > ⇢ Managing recording modules.
Inserting separator action
As a result, the recording is not only structured visually. The separator also raises a report issue with the separator header text as report information.
Separator report message
A separator owns the report level ‘Info’, a concept which is introduced in the next chapter.
User-defined log message
User-defined log messages are a method for inserting additional report message if needed. This method is mainly used as a debugging instrument and the objective is to generate a report message at a specific position during the test run.
Inserting a user-defined log message
Log messages are of action type ‘Report’ which is also be made visible through the action icon in column #1
The report type can be specified by opening the corresponding list
- Log = The contents of the log action, i.e. a text, or the contents of a defined variable is displayed in the report
- Screenshot = A screenshot of the linked repository item is taken in the moment of execution and stored within the report
- Snapshot = This option triggers Ranorex Spy to analyze and store the repository status of the corresponding linked repository item in the moment of execution and store it. A link in the report opens Ranorex Spy and displays the stored repository status
- Column #2 holds the log message content
- This can either be a constant value (i.e. text message), or a reference to a variable where data-based testing can be applied
- Column #5 outlines the corresponding error level
User-defined log message in report
User-defined log message in report