By default the test suite file is opened automatically.
As was already mentioned in Lesson 2: Ranorex Modules - Test Actions, it is recommended to separate a test case into reusable and parameterizable test automation modules. Within the Ranorex test suite you can not only easily create new test cases by combining already existing automation modules - you can also specify global parameters and data connectors to set up data-driven test cases.
Within the following lesson you will learn about:
Test Suite Editor
Add a New Test CaseYou can add a new test case by clicking the 'NEW' button. Depending on the current item selection of the test suite you are allowed to add different types of test suite elements. A test suite can be made up of the following items:
- Test cases
- Recording modules
- Code modules
- Module groups
Select the 'Test Case' item to create a new test case at the current position.
Note: Within a test suite, test cases can be added at different levels. A test case can also be added as a child to an existing test case or to a folder item.
The newly created test case will be used for validating the correctness of KeePass' version number so the name of the test case should be changed. Select and click on the test case and rename it to 'TC_ValidateVersionNumber'.
Reuse of Existing Modules
Now you are ready to reuse two existing record modules from the Ranorex Module Browser View.
You can simply drag and drop items from the module browser into the test case or you can add new or existing items using the context menu as shown.
Press 'OK' to add the record module to the test case.
Repeat the steps to add the record items 'LogIn' and 'CloseSUT' to the test case.
Insert a New Recording Module
For now the test case only covers starting the application under test, logging in and closing the application under test using three existing modules.
In order to validate the version number of the KeePass application, you can create a new recording module to be used in the test case.
Use the context menu again, but this time to insert a new recording into the test case.
Before starting the new recording, you should confirm that KeePass is running - you could start the application manually if it is not running.
The new recording module 'ValidateVersionNumber' should cover the following steps:
- open the about dialog
- validate the version number
- close the about dialog
Keep in mind that you need to select the 'Instant Recording' option because the system under test will be started by the reused recording module 'StartSUT'.
During the validation step, Ranorex automatically created a new repository item for the cell holding the version number from KeePass - the text itself is used for identification of the cell (path). Since the content of the cell (KeePass' version number) is used for identification, a higher version number would cause the test case to fail because the cell can't be found anymore - independent from the match of version number in the validation step itself. To avoid this, the path to this cell should be modified to be more robust. The path can be edited using the path editor. Additionally it should not use the version number itself for identification.
Now the test case 'ValidateVersionNumber' is ready to be executed.
Use the context menu item 'Run Selected Test Case' to run it and see whether it works or not.
Module Group Editor
For example combining the modules 'StartSUT' and 'LogIn' into a module group 'StartAndLogin' within the KeePass sample project could be useful, because these two modules are needed to start the system under test. The same applies to the modules 'Save' and 'CloseSUT' which can be combined with the module group 'SaveAndClose'.
This can be done using the module group editor.
In addition to the test suite file, the test suite project also contains a file defining the module groups existing in the project. This module group file is generated automatically as <Project-Name>.rxtmg.
Double-clicking this file in the project view will open it in the module group editor.
- Press the 'New Module Group' button in the module group editor
- Use the context menu in the test suite editor to add a module group to a test case
- Use the context menu in the test suite editor to group selected modules into a module group
Next to adding existing modules it is possible to add folders to module groups to add additional hierarchical levels.
An existing module can also be added using drag and drop from the module browser or project view into the module group in the module group editor.
You can define data binding for modules in the module group by using the context menu item 'Data Binding...' which appears by right-clicking a module group.
The data binding dialog allows you to specify which of the module variables should be changeable from outside the module group and which of the module variables should have constant values inside the module group.
Note: The mechanism for using constant values within module groups allows hiding module variables. This can reduce the complexity and increases the clarity of module groups because it is not always necessary to set each module variable individually.
The data binding for module groups works exactly the same way as it works for modules.
General Structure of a Test Suite and its Test Cases
To see the different ways of organizing a more complex test suite project, open the sample test suite project 'KeePassTestSuite' from the Ranorex Studio start page.
|#1 Folder||Used to group multiple test cases|
|#2 Test case||Represents a test case which can contain modules, a setup or teardown region or other test cases|
#3 Setup region
|Groups modules used to prepare a test case (e.g. start system under test, initialize a database, etc.)|
|#4 Module group||Used to group several modules into a reusable set|
#5 Teardown region
|Groups modules used to clean up a test case (e.g. deleting files generated during test execution, closing application, etc.)|
#6 Code module
|Automation module written in code|
|#7 Recording module||Automation module generated by recording|
Setup and Teardown Regions
The setup and teardown regions are used to prepare and clean up a single test case run.
The setup region will be executed before any other module held by the test case and should hold any modules needed to bring the system under test in exactly the state your test can start from. A typical application of this section is to start the application under test and log in with a user.
The teardown region will be executed when the execution of the test case has been finished, which means after all modules have been executed, or an error has aborted the test case. The teardown region should hold any modules needed to clean up the system under test and bring it to the original state. A typical application of this section is to delete all added data and close the application under test.
The setup region will automatically be placed at the beginning and the teardown region will automatically be placed at the end of a test case.
Use the context menu to specify which modules or module groups from a test case should be part of the setup or teardown process.
Note: If you want to define one setup and one teardown region for a set of test cases, simply nest your test cases as shown in the following picture
Note: In order to define global setup and teardown regions click on 'Show Setup/Teardown' in context menu of the test suite node and drag and drop the wanted modules or module groups to displayed the setup and teardown regions.
In order to quickly deactivate a certain module, instead of deleting it from the test case, use the context menu item 'Disable'.
Searching For Elements
Use the 'Search' text box to find elements in the Test Suite.
Running a Test Suite - Run Configurations
To run a test suite click the 'RUN' button shown in the test suite editor's toolbar. Use the check boxes to specify whether single test cases or a group of test cases should be part of a test suite run. Create different test suite run configurations using the drop-down box from the toolbar.
These test suite run configurations define if a specific test case will be executed or not.
For example one test suite run configuration can be defined where the mobile tests stored in your test suite will be executed and one test suite run configuration can be defined where they won't.
Running Tests without Ranorex Studio
As you already learned in Lesson 1: Getting Started, Ranorex Studio creates an executable file from your test suite project. In order to execute the test suite in a runtime environment, you have to have the generated executable (*.EXE) and the test suite file (*.RXTST) in the same directory. If your Ranorex Studio solution consists of more than one project, you need to ensure that the library (*.DLL) files are also part of the same directory. In short, to deploy a Ranorex test to a runtime machine it's required to copy the complete output folder (e.g. 'bin/debug') to the target machine.
You can execute the test suite outside Ranorex Studio using one of the following:
Ranorex Test Suite Runner
Simply double-click the *.RXTST file from the project's output folder to open the Ranorex Test Suite Runner.
Additionally one can create new run configurations the same way as is done within a Ranorex Studio Project.
Running Tests via Command Line
Note: When executing a test suite from command line the return value '0' signals the successful execution of the test script and the return value '-1' signals a failure.
Using the following, you can execute the test suite from the command line:
Prints this help text.
Lists all settable configuration parameters and their values.
config|cfg:<config parameter name>=<value>
Set values for configuration parameters.
reportfile|rf:<report file path>
Sets the name (and path) of the report file. If no path is provided, the current directory is used. By default, the filename specified in the rxtst file is used. (for example: %S_%Y%M%D_%T.rxlog).
Compresses the report (including associated files) to a single archive (".rxzlog" extension).
zipreportfile|zrf:<zip report file path>
When used with /zipreport|zr, sets the name (and path) of the compressed report file. If no path is provided, the path of the report file is used. If the file extension is not ".rxzlog", the extension will be replaced with ".rxzlog". By default, the report filename specified in the rxtst file or the value of reportfile|rf is used with an ".rxzlog" extension (for example: %S_%Y%M%D_%T.rxzlog).
reportlevel|rl: Debug|Info|Warn|Error|Success|Failure|<any integer>
Sets the minimum report level that log messages need to have in order to be included in the log file. Specify 'None' to completely disable reporting. These levels correspond to the following integer values: Debug=10,Info=20,Warn=30,Error=40,Success=110,Failure=120
Lists all global parameters and their values.
listtestcaseparams|ltcpa:<name or guid of test case>
Lists all testcase parameters and their values.
testcase|tc:<name or guid of test case>
Runs this test case only.
testsuite|ts:<path to test suite file>
Runs the test cases defined by the test suite (rxtst) file. By default
- the rxtst file with the same name as the <TestSuiteExe> is used
- or the first rxtst file in the same folder as <TestSuiteExe>.
runconfig|rc:<configuration name>Runs the test cases of the specified configuration defined by the rxtst file. Configurations can be edited using Ranorex Studio or TestSuiteRunner. By default, the currently selected run config is used.
module|mo:<module name or guid>
Runs the module with the specified name or guid. Assemblies loaded by <TestSuiteExe> and assemblies referenced in the rxtst file are searched.
param|pa:<global parameter name>=<value>
Creates or overrides values for global parameters specified in the test suite.
testcaseparam|tcpa:<name or guid of test case>:<parameter name>=<value>
Creates or overrides values for testcase parameters specified in the test suite.
Sets a custom runlabel for the test run.
testcasedatarange|tcdr:<name or guid of test case>=<data range>
Sets the data range for a testcase.
Note: Compressed report files (*.rxzlog) can be extracted by right-clicking them in explorer and choosing 'Extract' from context menu. Report files (*.rxlog) can be compressed by right-clicking them in explorer and choosing 'Compress' from context menu.
Some Examples for using Command Line Arguments
Using the KeePass sample test suite - included in Ranorex installation - as reference, the following samples will fulfill the following use cases:
Start the test suite with setting a global parameter:
Start only a specific test case of the test suite:
Start the test suite with generating a zipped report called report.rxzlog:
KeePassTestSuite.exe /zr /zrf:report.rxzlog
Start a specific run configuration of the test suite:
Start the test suite with specific data range
Test Suite Settings
General Test Suite Settings
|Name||Specifies the name of the test suite (same as shown within the test suite editor)|
|Description||Description of the test suite (same as shown within the description column of the test suite editor). Use the HTML text editor to format the description and to add links to the description. The formatted description will be represented as HTML content in the generated report|
Show Progress Dialog
|Specifies whether a progress dialog should be shown during test suite execution|
|Report Level||Specifies the minimum report level of messages shown with the report file|
|Warn for unbound variables||Specifies whether a warning should be shown for unbound variables|
Globally specified parameters can be accessed and used by every test case within the test suite. The KeePassTestSuite example project uses a global parameter to specify the directory path for the application under test. You can easily connect global parameters with variables as shown in the test case 'TC_AddEntry'. In addition you can use the global parameter to transfer values or states between test cases.
Additional Report Settings
|Enable Report||Specifies whether a test report should be generated|
|Report File Directory||Specifies the directory for the generated report files|
|Report File||Specifies the filename generated by the report; the following placeholders are available:
%L Run Label
%C Run Config Name
%H Host Name
%S Test Suite Name
%X Test Suite Result
|Auto Save Interval||Specifies how often the report file is saved during execution|
|Timestamp||Specifies whether the timestamp should be computed relative to the test suite start time or relative to the test module start time or the machine time should be taken|
|Compressed Copy||Specifies whether a copy of the report should be generated as compressed folder|
|Collect screenshots in a subdirectory||Specifies whether the created screenshots will be stored in a specific folder for each report or directly in the report folder|
|Report Template||Specifies the directory holding customized style files used instead of the Ranorex default style to present the reports; a new template can be created by pressing the button 'Create Custom Template' as explained in 'Lesson 8: Reporting - Create a Custom Report Template'; an existing template can be chosen by pressing the button 'Choose Custom Template'|
|Tracing Screenshot Mode||Specifies whether the screenshots will be captured in foreground (before action is executed) or background (while action is being executed) or capturing tracing screenshots is disabled|
|Quality||Specifies the image quality of the captured screenshots|
Test Case Settings
The 'General' tab of the test case's properties dialog is mainly used to setup how a broken test case impacts other test cases within the test suite.
|Name||Name of the test case|
|Description||Description of the test case (same as shown within the description column of the test suite editor). Use the HTML text editor to format the description and to add links to the description. The formatted description will be represented as HTML content in the generated report|
|Report Level||Specifies the level of messages written to the report file|
|Specifies the behavior of the test case and the test suite in case of on error. For further information please have a look at the next paragraph.|
The following figures try to illustrate several Error Behavior settings:
- Set to 'Continue with iteration' if, in case of an error, the case should not break continue with the next iteration
- Set to 'Continue with sibling' if, in case of an error, the currently running test case should break and the execution should continue with the sibling test case
- Set to 'Continue with parent' if, in case of an error, the currently running test case should break and the execution should continue with the sibling of the parent test case
- Set to 'Stop' in order to break the test suite run in case of an error
Using Data in Test Suites
As already explained in Lesson 3: Data-Driven Testing you can make your automation data-driven by using variables in your modules (recordings, code modules) as well as in your repositories.
These variables can be connected to internal or external data sources and parameters.
Different kinds of Data Container
Variables are the interface between a module and the nesting test case, or - if module groups are used - the module group, respectively.
Variables can be used in
Constant values can be used in module groups to hide module variables outside a module group. This can help to reduce the complexity and increases the clarity of module groups.
For further details have a look at the section about the Module Group Editor.
In module groups, group variables are the interface between the nested modules and the nesting test case.
For further details have a look at the section about the Module Group Editor.
The columns of a data connector are called Data Columns and specify the columns from external data sources.
Have a look at the data connectors section (Data Connectors) to get an overview about the different kinds of data connectors. Data columns can be connected to variables in the data binding dialog of a test case as described in section Combining Variables with Test Data.
Module variables can also be bound to parameters. A distinction is made between local parameters, which are part of a specific test case and are available in the test case and all child test cases, and global parameters which are available in the whole test suite.
Unlike local parameters, global parameters can be set from the command line as explained in section Running Tests via Command Line.
Have a look at the section Combining Variables with Parameters to see how parameters can be connected to variables.
Scope of Data Container
Global parameters are available in the whole test suite. This means you can bind global parameters in any test case of a test suite.
Local parameters and data columns will be inherited by all children of a test case. This means you can bind all local parameters and data columns of a test case in its child test cases.
The following figure will illustrate the scope of different data containers.
The green rectangle illustrates the scope of global parameters, the blue rectangle the scope of local parameters and data columns of the test case named 'Parent_Test_Case_1' and the red rectangle the scope of local parameters and data columns of the test case named 'Test_Case_1'.
This means, in 'Test_Case_1' you can bind all global parameters as well as all local parameters and data columns of itself and its parent test case 'Parent_Test_Case_1', as illustrated in the following figure.
Modify Parameter and Data Column Values
Parameter values as well as data column values can be changed by a module. In order to transfer values from one module to another, parameter or data column values are updated from their bound module variables once the module has finished execution.
This means you can freely transfer values within the scope of a parameter or data column.
One example of this application would be to dynamically get a value from the UI which should be reused in another module as described in the section Lesson 5: Ranorex Recorder - Additional Editing Options - Types of Action Items addressing the Get Value action or at Code Examples - Accessing Test Case & Test Suite Context.
- Ranorex Studio - The Layout
- Lesson 1: Getting Started
- Lesson 2: Ranorex Modules - Test Actions
- Lesson 3: Data-Driven Testing
- Lesson 4: Ranorex Test Suite
- Lesson 5: Ranorex Recorder
- Lesson 6: UI Mapping with Ranorex Repository
- Lesson 7: Code Modules
- Lesson 8: Reporting
- Lesson 9: Ranorex Spy
- General Ranorex Settings
- Code Examples
- Data Connectors
- Instrumentation Wizard
- Working with 2.X Projects in Ranorex Test Suites
- Technology Instrumentation
- RanoreXPath Weight Rule Library
- Ranorex UI Adapter
- Mobile Testing
- Android Testing
- iOS Testing
- Web Testing
- Source Control
- Ranorex Studio IDE
- Visual Studio Integration
- System Requirements
- 64-bit Platforms
- Remotely Working with Ranorex
- Silent Installation of Ranorex
- XCOPY Deployment