- Automate Testing of Desktop, Web and Mobile Software
As was already mentioned in Lesson 2: Ranorex Modules - Test Actions, it is recommended to separate a test case into reusable and parameterizable test automation modules. Within the Ranorex test suite you can not only easily create new test cases by combining already existing automation modules - you can also specify global parameters and data connectors to set up data-driven test cases.
Within the following lesson you will learn about:
Select the 'Test Case' item to create a new test case at the current position.
Note: Within a test suite, test cases can be added at different levels. A test case can also be added as a child to an existing test case or to a folder item.
Press 'OK' to add the record module to the test case.
Repeat the steps to add the record items 'LogIn' and 'CloseSUT' to the test case.
For now the test case only covers starting the application under test, logging in and closing the application under test using three existing modules.
In order to validate the version number of the KeePass application, you can create a new recording module to be used in the test case.
Use the context menu again, but this time to insert a new recording into the test case.
Before starting the new recording, you should confirm that KeePass is running - you could start the application manually if it is not running.
The new recording module 'ValidateVersionNumber' should cover the following steps:
Keep in mind that you need to select the 'Global Recording' option because the system under test will be started by the reused recording module 'StartSUT'.
During the validation step, Ranorex automatically created a new repository item for the cell holding the version number from KeePass - the text itself is used for identification of the cell (path). Since the content of the cell (KeePass' version number) is used for identification, a higher version number would cause the test case to fail because the cell can't be found anymore - independent from the match of version number in the validation step itself. To avoid this, the path to this cell should be modified to be more robust. The path can be edited using the path editor. Additionally it should not use the version number itself for identification.
Now the test case 'ValidateVersionNumber' is ready to be executed.
Use the context menu item 'Run Selected Test Case' to run it and see whether it works or not.
For example combining the modules 'StartSUT' and 'LogIn' into a module group 'StartAndLogin' within the KeePass sample project could be useful, because these two modules are needed to start the system under test. The same applies to the modules 'Save' and 'CloseSUT' which can be combined with the module group 'SaveAndClose'.
This can be done using the module group editor.
In addition to the test suite file, the test suite project also contains a file defining the module groups existing in the project. This module group file is generated automatically as <Project-Name>.rxtmg.
Double-clicking this file in the project view will open it in the module group editor.
You can define data binding for modules in the module group by using the context menu item 'Data Binding...' which appears by right-clicking a module group.
The data binding dialog allows you to specify which of the module variables should be changeable from outside the module group and which of the module variables should have constant values inside the module group.
Note: The mechanism for using constant values within module groups allows hiding module variables. This can reduce the complexity and increases the clarity of module groups because it is not always necessary to set each module variable individually.
The data binding for module groups works exactly the same way as it works for modules.
To see the different ways of organizing a more complex test suite project, open the sample test suite project 'KeePassTestSuite' from the Ranorex Studio start page.
|#1 Folder||Used to group multiple test cases|
|#2 Test case||Represents a test case which can contain modules, a setup or teardown region or other test cases|
#3 Setup region
|Groups modules used to prepare a test case (e.g. start system under test, initialize a database, etc.)|
|#4 Module group||Used to group several modules into a reusable set|
#5 Teardown region
|Groups modules used to clean up a test case (e.g. deleting files generated during test execution, closing application, etc.)|
#6 Code module
|Automation module written in code|
|#7 Recording module||Automation module generated by recording|
The setup and teardown regions are used to prepare and clean up a single test case run.
The setup region will be executed before any other module held by the test case and should hold any modules needed to bring the system under test in exactly the state your test can start from. A typical application of this section is to start the application under test and log in with a user.
The teardown region will be executed when the execution of the test case has been finished, which means after all modules have been executed, or an error has aborted the test case. The teardown region should hold any modules needed to clean up the system under test and bring it to the original state. A typical application of this section is to delete all added data and close the application under test.
The setup region will automatically be placed at the beginning and the teardown region will automatically be placed at the end of a test case.
Use the context menu to specify which modules or module groups from a test case should be part of the setup or teardown process.
In order to quickly deactivate a certain module, instead of deleting it from the test case, use the context menu item 'Disable'.
To run a test suite click the 'RUN' button shown in the test suite editor's toolbar. Use the check boxes to specify whether single test cases or a group of test cases should be part of a test suite run. Create different test suite run configurations using the drop-down box from the toolbar.
These test suite run configurations define if a specific test case will be executed or not.
For example one test suite run configuration can be defined where the mobile tests stored in your test suite will be executed and one test suite run configuration can be defined where they won't.
As you already learned in Lesson 1: Getting Started, Ranorex Studio creates an executable file from your test suite project. In order to execute the test suite in a runtime environment, you have to have the generated executable (*.EXE) and the test suite file (*.RXTST) in the same directory. If your Ranorex Studio solution consists of more than one project, you need to ensure that the library (*.DLL) files are also part of the same directory. In short, to deploy a Ranorex test to a runtime machine it's required to copy the complete output folder (e.g. 'bin/debug') to the target machine.
You can execute the test suite outside Ranorex Studio using one of the following:
Additionally one can create new run configurations the same way as is done within a Ranorex Studio Project.
Note: When executing a test suite from command line the return value '0' signals the successful execution of the test script and the return value '-1' signals a failure.
Using the following, you can execute the test suite from the command line:
Prints this help text.
Lists all settable configuration parameters and their values.
Set values for configuration parameters.
Sets the name (and path) of the report file. If no path is provided, the current directory is used. By default, the filename specified in the rxtst file is used. (for example: %S_%Y%M%D_%T.rxlog).
Compresses the report (including associated files) to a single archive (".rxzlog" extension).
When used with /zipreport|zr, sets the name (and path) of the compressed report file. If no path is provided, the path of the report file is used. If the file extension is not ".rxzlog", the extension will be replaced with ".rxzlog". By default, the report filename specified in the rxtst file or the value of reportfile|rf is used with an ".rxzlog" extension (for example: %S_%Y%M%D_%T.rxzlog).
Sets the minimum report level that log messages need to have in order to be included in the log file. Specify 'None' to completely disable reporting. These levels correspond to the following integer values: Debug=10,Info=20,Warn=30,Error=40,Success=110,Failure=120
Lists all global parameters and their values.
Lists all testcase parameters and their values.
Runs this test case only.
Runs the test cases defined by the test suite (rxtst) file. By default
Runs the module with the specified name or guid. Assemblies loaded by <TestSuiteExe> and assemblies referenced in the rxtst file are searched.
Creates or overrides values for global parameters specified in the test suite.
Creates or overrides values for testcase parameters specified in the test suite.
Sets a custom runlabel for the test run.
Sets the data range for a testcase.
Note: Compressed report files (*.rxzlog) can be extracted by right-clicking them in explorer and choosing 'Extract' from context menu. Report files (*.rxlog) can be compressed by right-clicking them in explorer and choosing 'Compress' from context menu.
Using the KeePass sample test suite - included in Ranorex installation - as reference, the following samples will fulfill the following use cases:
KeePassTestSuite.exe /zr /zrf:report.rxzlog
|Name||Specifies the name of the test suite (same as shown within the test suite editor)|
|Description||Description of the test suite (same as shown within the description column of the test suite editor). Use the HTML text editor to format the description and to add links to the description. The formatted description will be represented as HTML content in the generated report|
Show Progress Dialog
|Specifies whether a progress dialog should be shown during test suite execution|
|Capture Tracing Screenshots||Specifies whether tracing screenshots should be captured or not in the case of an error|
|Report Level||Specifies the minimum report level of messages shown with the report file|
|Warn for unbound variables||Specifies whether a warning should be shown for unbound variables|
|Report Enabled||Specifies whether a test report should be generated|
|Report File Directory||Specifies the directory for the generated report files|
|Report File||Specifies the filename generated by the report; move the mouse over the 'Description' label to see available placeholder values|
|Compressed Copy||Specifies whether the report should be generated as compressed folder|
|Report Template Folder||Specifies the directory holding customized style files used instead of the Ranorex default style to present the reports; a new template can be created by pressing the button 'Create Custom Template' as explained in 'Lesson 8: Reporting - Create a Custom Report Template'|
Auto Save Interval
|Specifies how often the report file is saved during execution|
|Place screenshots in folder||Specifies whether the created screenshots will be stored in a specific folder for each report or directly in the report folder|
Globally specified parameters can be accessed and used by every test case within the test suite. The KeePassTestSuite example project uses a global parameter to specify the directory path for the application under test. You can easily connect global parameters with variables as shown in the test case 'TC_AddEntry'. In addition you can use the global parameter to transfer values or states between test cases.
The 'General' tab of the test case's properties dialog is mainly used to setup how a broken test case impacts other test cases within the test suite.
|Name||Name of the test case|
|Description||Description of the test case (same as shown within the description column of the test suite editor). Use the HTML text editor to format the description and to add links to the description. The formatted description will be represented as HTML content in the generated report|
|Report Level||Specifies the level of messages written to the report file|
|Specifies the behavior of the test case and the test suite in case of on error. For further information please have a look at the next paragraph.|
The following figures try to illustrate several Error Behavior settings:
As already explained in Lesson 3: Data-Driven Testing you can make your automation data-driven by using variables in your modules (recordings, code modules) as well as in your repositories.
These variables can be connected to internal or external data sources and parameters.
Variables are the interface between a module and the nesting test case, or - if module groups are used - the module group, respectively.
Variables can be used in
Constant values can be used in module groups to hide module variables outside a module group. This can help to reduce the complexity and increases the clarity of module groups.
For further details have a look at the section about the Module Group Editor.
In module groups, group variables are the interface between the nested modules and the nesting test case.
For further details have a look at the section about the Module Group Editor.
The columns of a data connector are called Data Columns and specify the columns from external data sources.
Have a look at the data connectors section (Data Connectors) to get an overview about the different kinds of data connectors. Data columns can be connected to variables in the data binding dialog of a test case as described in section Combining Variables with Test Data.
Module variables can also be bound to parameters. A distinction is made between local parameters, which are part of a specific test case and are available in the test case and all child test cases, and global parameters which are available in the whole test suite.
Unlike local parameters, global parameters can be set from the command line as explained in section Running Tests via Command Line.
Have a look at the section Combining Variables with Parameters to see how parameters can be connected to variables.
Global parameters are available in the whole test suite. This means you can bind global parameters in any test case of a test suite.
Local parameters and data columns will be inherited by all children of a test case. This means you can bind all local parameters and data columns of a test case in its child test cases.
The following figure will illustrate the scope of different data containers.
The green rectangle illustrates the scope of global parameters, the blue rectangle the scope of local parameters and data columns of the test case named 'Parent_Test_Case_1' and the red rectangle the scope of local parameters and data columns of the test case named 'Test_Case_1'.
This means, in 'Test_Case_1' you can bind all global parameters as well as all local parameters and data columns of itself and its parent test case 'Parent_Test_Case_1', as illustrated in the following figure.
Parameter values as well as data column values can be changed by a module. In order to transfer values from one module to another, parameter or data column values are updated from their bound module variables once the module has finished execution.
This means you can freely transfer values within the scope of a parameter or data column.
One example of this application would be to dynamically get a value from the UI which should be reused in another module as described in the section Lesson 5: Ranorex Recorder - Additional Editing Options - Types of Action Items addressing the Get Value action or at Code Examples - Accessing Test Case & Test Suite Context.