By default, each run of a test suite, test case or single module generates a report file (*.rxlog) which tells you if the run was successful or not. In the following lesson you will learn about the following:
The report shown after running a test suite provides a general overview of how many of the test cases have been executed successfully, have failed or have been blocked. Each executed test case and all its child modules can be analyzed in depth by expanding a particular test case.
In addition to that, a test suite report also informs you about the following:
The detailed log messages generated by a recording or by a code module shown in the figure above consist of a time stamp, a report level, a category and message text. By default logging is turned on for recordings. Use the Recorder's settings dialog to change the default value and to turn off logging for new recordings. In order to turn logging off or on for a particular action item, open the properties and set the 'Use Default Logging' attribute to false.
Expand the report as shown above and move the mouse pointer over the first log message.
Click at 'Jump to Item' to open the module and to select the correspondending action. Use the quick link 'Open in Spy' to analyze the RanoreXPath expression used for the particular action item. This is especially useful in situations when a test run fails with the message 'Failed to find item...' error message. Move the mouse cursor over a testcase and click on 'Jump to Testcase' to select the correspondending test case in the test suite.
For each run of a test suite, a single test case or a recording a new log file (*.rxlog) is generated and saved in the project's 'Report' folder. You can open older report files by double-clicking the file in the project view. Specify the file names used for the report files within the test suite's settings dialog.
Note: It is not required to run Ranorex Studio to open a report file. You can also open the report file from Windows Explorer. If you're going to copy or send the file by email, it is recommended to use the zipped report as described in "Lesson 4: Ranorex Test Suite - Test Suite Settings".
Within Ranorex you are able to use different levels of reporting information importance, from a debug message to a failure information. In addition to predefined report levels, you can also easily define your own levels.
Report Levels describe the importance of a message that is delivered to the report. Using different kinds of report levels can help you keep your reports short and better readable.
Within a test suite or even within a test case, you are able to define a report level that describes the minimum level of reporting information (see Lesson 4: Ranorex Test Suite - Test Suite Settings for more information). Report information with a lower level of importance is ignored and does not fill up your report, whereas decreasing a test case's report level could assist you in finding errors in your testing scenario.
Ranorex provides the following predefined report levels - the level itself is indicated by integer values (which are written in parenthesis):
The first five items describe different levels of importance for report messages. For instance an error message in the report indicates a logical test step error whereas a warning message should only get your attention but not indicate an error.
From report levels 'Debug' to 'Success', the result of the current module is still classified as a success whereas the 'Failure' report level classifies the module result as a failure; this is indicated by a red mark in the report.
Note: A test case's failure depends on the success of its child elements which can either be modules or nested test cases. A module fails when an exception is thrown or a failure message is being logged. Exceptions are typically thrown by failed validation actions, by searching for nonexistent elements or by an explicit call at code level.
In addition to predefined report levels, you are able to define your own report level with a custom importance value. You can do that either from code or within the Recorder (see Logging Individual Information for detailed information).
Individual information can be sent to the report either by using usercode (i.e. code modules) or by using the recorder action 'Report'.
You can use one of the following methods from the Ranorex 'Report' class in order to report information at a specific level:
These methods can be used with a single parameter, the message text. Alternatively they can be used with an additional parameter indicating the category which is set to the default value 'User' in the first overload of such methods. The default category can also be set via code by assigning the 'Ranorex.Report.DefaultCategory' property. The category can be seen in a distinct column in the report. The following code lines demonstrate different reporting methods (and different reporting categories), resulting in the report shown after the code example.
// Reporting information (debug, info) using default Category "User" Ranorex.Report.Debug("This is a Debug Information"); Ranorex.Report.Info("This is a Information"); // Setting Parameter Category to specific value and report a warning and an error Ranorex.Report.Warn("Specific Category","This is a Warning Information"); Ranorex.Report.Error("Specific Category", "This is an Error Information"); // Setting the Default Category Ranorex.Report.DefaultCategory = "My new default category"; // Reporting information (success, failure) using the new default category Ranorex.Report.Success("This is a success information"); Ranorex.Report.Failure("This is a failure Information");
' Reporting information (debug, info) using default Category "User" Ranorex.Report.Debug("This is a Debug Information") Ranorex.Report.Info("This is a Information") ' Setting Parameter Category to specific value and report a warning and an error Ranorex.Report.Warn("Specific Category", "This is a Warning Information") Ranorex.Report.Error("Specific Category", "This is an Error Information") ' Setting the Default Category Ranorex.Report.DefaultCategory = "My new default category" ' Reporting information(success, failure) using the new default category Ranorex.Report.Success("This is a success information") Ranorex.Report.Failure("This is a failure Information")
Note: Please consider that the report information at the failure level provides a screenshot automatically (if tracing screenshots is enabled). In order to visually retrace the last steps that lead to the current error, screenshots will be provided for the last three actions.
You are able to define your own report levels with a custom name and a value for the level:
Ranorex.ReportLevel MyNewReportLevel = new ReportLevel("My low Report Level", 25, null); Ranorex.Report.Log (MyNewReportLevel, "This is unimportant information");
Dim MyNewReportLevel As Ranorex.ReportLevel = New ReportLevel("My low Report Level", 25, Nothing) Ranorex.Report.Log(MyNewReportLevel, "This is unimportant information")
Depending on the current Report Level (see next chapter), the report information with the user-defined report level should appear in the report as follows:
Report information only appears in the report if the parent test case has a report level setting lower than or equal to the current report information level. The current report level can also be set using code:
Ranorex.Report.CurrentReportLevel = Ranorex.ReportLevel.Parse("Only Highest Importance;90");
"Ranorex.Report.CurrentReportLevel = Ranorex.ReportLevel.Parse("Only Highest Importance;90")
This assignment causes the inclusion of reporting information only at a level greater the or equal to 90, which is 'Success' and 'Failure'. The resulting report, which is caused by the same code from above but with a different current report level, can be seen below.
Note: The current report level can always be overridden by logging information with the report level 'Always'.
Ranorex.Report.Log (Ranorex.ReportLevel.Always, "User", "Any-Case message");
Ranorex.Report.Log (Ranorex.ReportLevel.Always, "User", "Any-Case message")
Screenshots of the current state of the system under test or any UI element can be easily sent to the report.
If you call the screenshot method without any arguments, a screenshot of the whole desktop will be available in your report.
Note: Screenshots from UI elements can only be made if the element is visible at the time a screenshot is made. Controlling your application using code does not necessarily bring your application to the front and make it visible. To ensure the visibility of UI elements, use the 'EnsureVisible' method which is provided by any element that is derived from Ranorex Adapter class (e.g. 'Text', 'RadioButton', 'Button', etc.). Consequently, rooted folders from repository or even application folders do not ensure visibility. Alternatively you can ensure visibility by using the 'Self' property from that object types (rooted folder or application folder).
The Ranorex reporting class provides a way to simply report a system summary:
The Ranorex Recorder provides a way to send information directly to the report. Just use the 'Add New Action' button from the toolbar and add a 'Report' Action.
Just like a call from code, you can specify one of the predefined report levels directly within the recorder's actions table.
Report messages with a custom report level can also be submitted. Simply choose the 'Custom' report level in the drop-down.
Since Ranorex Version 3.3, you can send screenshots and even snapshots to your report directly from the recorder's actions table. These two actions require a connection to a repository item in order to specify from which object the screenshot or snapshot should be made.
With Ranorex 4.0.0 the custom report format has been changed from one single file to a template folder holding all necessary files. After upgrading to Ranorex 4 and opening a solution using custom reports, a warning will appear in the Error pane.
A new report template folder has automatically been created at
The newly created files can be shown in project view by pressing the 'Show All Files' button.
Note: Press the 'Refresh' button to ensure that the new created folder will be shown.
Open the file 'RanorexReport4.css' to edit the style of the report according to your wishes.
Note: It's recommended to make customizations at the end of the file.
Open the file 'RanorexReport4.xsl' to edit the structure of the report according to your wishes.
Open the file 'View.rxlog' to get a preview of your customizations using some sample data.
Download Test Automation Guide
(PDF file, 20MB)