As was already mentioned in Lesson 2: Ranorex Modules - Test Actions, it is recommended to separate a test case into reusable and parameterizable test automation modules. Within the Ranorex test suite you can not only easily create new test cases by combining already existing automation modules - you can also specify global parameters and data connectors to set up data-driven test cases.
Within the following lesson you will learn about:
Test Suite Editor
By default the test suite file is opened automatically.
Adding a new test case or smart folderYou can add a new test case or smart folder by clicking the 'ADD' button. Depending on the current item selection of the test suite you are allowed to add different types of test suite elements. A test suite can be made up of the following items:
- Test cases
- Smart folders
- Recording modules
- Code modules
- Module groups
These items can be added and arranged according to a special hierarchy. You can find out more about it here.
The newly created test case will be used for validating the correctness of KeePass' version number so the name of the test case should be changed. Select and click on the test case and rename it to 'TC_ValidateVersionNumber'.
Reuse of Existing Modules
Now you are ready to reuse two existing record modules from the Ranorex Module Browser View.
You can simply drag and drop items from the module browser into the test case or you can add new or existing items using the context menu as shown.
Press 'OK' to add the record module to the test case.
Repeat the steps to add the record items 'LogIn' and 'CloseSUT' to the test case.
Insert a New Recording Module
For now the test case only covers starting the application under test, logging in and closing the application under test using three existing modules.
In order to validate the version number of the KeePass application, you can create a new recording module to be used in the test case.
Use the context menu again, but this time to insert a new recording into the test case.
Before starting the new recording, you should confirm that KeePass is running - you could start the application manually if it is not running.
The new recording module 'ValidateVersionNumber' should cover the following steps:
- open the about dialog
- validate the version number
- close the about dialog
Keep in mind that you need to select the 'Instant Recording' option because the system under test will be started by the reused recording module 'StartSUT'.
During the validation step, Ranorex automatically created a new repository item for the cell holding the version number from KeePass - the text itself is used for identification of the cell (path). Since the content of the cell (KeePass' version number) is used for identification, a higher version number would cause the test case to fail because the cell can't be found anymore - independent from the match of version number in the validation step itself. To avoid this, the path to this cell should be modified to be more robust. The path can be edited using the path editor. Additionally it should not use the version number itself for identification.
Now the test case 'ValidateVersionNumber' is ready to be executed.
Use the context menu item 'Run Selected Test Case' to run it and see whether it works or not.
Module Group Editor
For example combining the modules 'StartSUT' and 'LogIn' into a module group 'StartAndLogin' within the KeePass sample project could be useful, because these two modules are needed to start the system under test. The same applies to the modules 'Save' and 'CloseSUT' which can be combined with the module group 'SaveAndClose'.
This can be done using the module group editor.
In addition to the test suite file, the test suite project also contains a file defining the module groups existing in the project. This module group file is generated automatically as <Project-Name>.rxtmg.
Double-clicking this file in the project view will open it in the module group editor.
- Press the 'New Module Group' button in the module group editor
- Use the context menu in the test suite editor to add a module group to a test case/smart folder
- Use the context menu in the test suite editor to group selected modules into a module group
Next to adding existing modules it is possible to add folders to module groups to add additional hierarchical levels.
An existing module can also be added using drag and drop from the module browser or project view into the module group in the module group editor.
You can define data binding for modules in the module group by using the context menu item 'Data Binding...' which appears by right-clicking a module group.
The data binding dialog allows you to specify which of the module variables should be changeable from outside the module group and which of the module variables should have constant values inside the module group.
Note: The mechanism for using constant values within module groups allows hiding module variables. This can reduce the complexity and increases the clarity of module groups because it is not always necessary to set each module variable individually.
Note: By pressing the button 'Auto-Create' a group variable for each unbound module variable will be generated based on the name of the module variable. By pressing the button 'Auto-Bind' unbound module variables will be bound to group variables having identical names.
The data binding for module groups works exactly the same way as it works for modules.
General structure of a test suite
To see the different ways of organizing a more complex test suite project, open the sample test suite project 'KeePassTestSuite' from the Ranorex Studio start page.
|#1 Test suite||Represents the entire test suite and is the root level of the structure|
|#2 Test case||Represents a test case|
#3 Setup region
|Groups modules used to prepare a test case (e.g. start system under test, initialize a database, etc.)|
|#4 Smart folder||Used to organize other items. Smart folders can be nested (a smart folder may contain other smart folders)|
|#5 Module group||Used to group several modules into a reusable set|
#6 Teardown region
|Groups modules used to clean up a test case (e.g. deleting files generated during test execution, closing application, etc.)|
#7 Code module
|Automation module written in code|
|#8 Recording module||Automation module generated by recording|
These items follow a hierarchy. This means that there are rules that define which items may be added directly to other items in the test suite structure. The table below illustrates this.
Test suite: At the root level, you may add test cases, smart folders, and setup/teardown regions.
Test cases: You may add smart folders, setup/teardown regions, modules and module groups.
Smart folders: You may add test cases*, other smart folders, setup/teardown regions, modules** and module groups**.
Setup/teardown: You may add modules and module groups.
*Only if the smart folder is not part of a test case.
**Only if the smart folder is part of a test case.
Note: If a drag/drop or copy/paste operation would create a nested test case, the affected test case will be converted into a smart folder instead. The resulting smart folder will work the same way as the original test case.
Setup and teardown regions
The setup and teardown regions are used to prepare and clean up a single test case run.
The setup region will be executed before any other module held by the test case and should hold any modules needed to bring the system under test in exactly the state your test can start from. A typical case for using this section is to start the application under test and log in with a user.
The teardown region will be executed when the execution of the test case has been finished, which means after all modules have been executed, or an error has aborted the test case. The teardown region should hold any modules needed to clean up the system under test and bring it to the original state. A typical case for using this section is to delete all added data and close the application under test.
The setup region will automatically be placed at the beginning and the teardown region will automatically be placed at the end of a test suite, test case, or smart folder.
Use the context menu to specify which modules or module groups from a test case should be part of the setup or teardown process.
Note: If you want to define one setup and one teardown region for a set of test cases, simply use a smart folder as shown in the following picture
Note: In order to define global setup and teardown regions click on 'Add Setup' or 'Add Teardown' in context menu of the test suite node and drag and drop the wanted modules or module groups to displayed the setup and teardown regions.
In order to quickly deactivate a certain module, instead of deleting it, use the context menu item 'Disable'.
Searching for elements
Use the 'Search' text box to find elements in the test suite.
Running a Test Suite
To run a test suite click the 'RUN' button shown in the test suite editor's toolbar or the run button in the Ranorex Studio toolbar. This will always run the test suite using the currently selected run configuration.
Run configurations define which test cases and smart folders will be executed during a test. Use the check boxes to specify which elements you want to be run. Ticking the checkbox at the test suite level will tick all others. You can also save your current configuration. Here's how:
- Check the boxes of the desired elements.
- Click the drop-down menu in the test suite editor's toolbar and select "Manage Run Configurations...".
- Click "Add".
- Enter a name and click OK.
- You can now apply the run configuration by selecting it from the drop-down menu.
Note: Run configurations can only be used with the test suite they were created in.
Run iterations let you run specific test cases or smart folders any amount of times during a test run. You can enable them as follows:
- Click on the test cases(s) or smart folder(s) you want to iterate.
- Go to "View" and click on "Properties" or press F4. The properties pad will appear.
- Enter your desired number of iterations in "Iteration Count".
The selected items will now be run the specified amount of times during test execution.
Running Tests without Ranorex Studio
As you already learned in Lesson 1: Getting Started, Ranorex Studio creates an executable file from your test suite project. In order to execute the test suite in a runtime environment, you have to have the generated executable (*.EXE) and the test suite file (*.RXTST) in the same directory. If your Ranorex Studio solution consists of more than one project, you need to ensure that the library (*.DLL) files are also part of the same directory. In short, to deploy a Ranorex test to a runtime machine it's required to copy the complete output folder (e.g. 'bin/debug') to the target machine.
You can execute the test suite outside Ranorex Studio using one of the following:
- Ranorex Test Suite Runner
- Command Line
Ranorex Test Suite Runner
Simply double-click the *.RXTST file from the project's output folder to open the Ranorex Test Suite Runner.
Additionally one can create new run configurations the same way as is done within a Ranorex Studio Project.
Running Tests via Command Line
Note: When executing a test suite from command line the return value '0' signals the successful execution of the test script and the return value '-1' signals a failure.
Using the following command pattern, you can execute the test suite from the command line:
Prints this help text.
Lists all settable configuration parameters and their values.
config|cfg:<config parameter name>=<value>
Set values for configuration parameters.
Display name of the endpoint to be used as the automation root during test execution. If no endpoint is specified 'localhost' is used.
reportfile|rf:<report file path>
Sets the name (and path) of the report file. If no path is provided, the current directory is used. By default, the filename specified in the rxtst file is used. (for example: %S_%Y%M%D_%T.rxlog).
Compresses the report (including associated files) to a single archive (".rxzlog" extension).
Creates the report in JUnit format as well. The report will be placed in the same location as the original Ranorex Report.
zipreportfile|zrf:<zip report file path>
When used with /zipreport|zr, sets the name (and path) of the compressed report file. If no path is provided, the path of the report file is used. If the file extension is not ".rxzlog", the extension will be replaced with ".rxzlog". By default, the report filename specified in the rxtst file or the value of reportfile|rf is used with an ".rxzlog" extension (for example: %S_%Y%M%D_%T.rxzlog).
reportlevel|rl: Debug|Info|Warn|Error|Success|Failure|<any integer>
Sets the minimum report level that log messages need to have in order to be included in the log file. Specify 'None' to completely disable reporting. These levels correspond to the following integer values: Debug=10,Info=20,Warn=30,Error=40,Success=110,Failure=120
Lists all global parameters and their values.
listtestcaseparams|ltcpa:<name or guid of test case or smart folder>
Lists all test case or smart folder parameters and their values.
testcase|tc:<name or guid of test case or smart folder>
Runs this test case or smart folder only.
testsuite|ts:<path to test suite file>
Runs the test cases or smart folders defined by the test suite (rxtst) file. By default
- the rxtst file with the same name as the <TestSuiteExe> is used
- or the first rxtst file in the same folder as <TestSuiteExe>.
runconfig|rc:<configuration name>Runs the test cases or smart folder of the specified configuration defined by the rxtst file. Configurations can be edited using Ranorex Studio or TestSuiteRunner. By default, the currently selected run config is used.
module|mo:<module name or guid>
Runs the module with the specified name or guid. Assemblies loaded by <TestSuiteExe> and assemblies referenced in the rxtst file are searched.
param|pa:<global parameter name>=<value>
Creates or overrides values for global parameters specified in the test suite.
testcaseparam|tcpa:<name or guid of test case or smart folder>:<parameter name>=<value>
Creates or overrides values for test case or smart folder parameters specified in the test suite.
Sets a custom runlabel for the test run.
testcasedatarange|tcdr:<name or guid of test case or smart folder>=<data range>
Sets the data range for a test case or smart folder.
Note: Compressed report files (*.rxzlog) can be extracted by right-clicking them in explorer and choosing 'Extract' from context menu. Report files (*.rxlog) can be compressed by right-clicking them in explorer and choosing 'Compress' from context menu.
Some Examples for using Command Line Arguments
Using the KeePass sample test suite - included in Ranorex installation - as reference, the following samples will fulfill the following use cases:
Start the test suite with setting a global parameter:
Start only a specific test case of the test suite:
Start the test suite with generating a zipped report called report.rxzlog:
KeePassTestSuite.exe /zr /zrf:report.rxzlog
Start a specific run configuration of the test suite:
Start the test suite with specific data range
Test Suite Settings
General Test Suite Settings
|Name||Specifies the name of the test suite (same as shown within the test suite editor)|
|Description||Description of the test suite (same as shown within the description column of the test suite editor). Use the HTML text editor to format the description and to add links to the description. The formatted description will be represented as HTML content in the generated report|
Show Progress Dialog
|Specifies whether a progress dialog should be shown during test suite execution|
|Report Level||Specifies the minimum report level of messages shown with the report file|
|Warn for unbound variables||Specifies whether a warning should be shown for unbound variables|
Globally specified parameters can be accessed and used by every test case within the test suite. The KeePassTestSuite example project uses a global parameter to specify the directory path for the application under test. You can easily connect global parameters with variables as shown in the test case 'TC_AddEntry'. In addition you can use the global parameter to transfer values or states between test cases.
Additional Report Settings
|Enable Report||Specifies whether a test report should be generated|
|Report File Directory||Specifies the directory for the generated report files|
|Report File||Specifies the filename generated by the report; the following placeholders are available:
%L Run Label
%C Run Config Name
%H Host Name
%S Test Suite Name
%X Test Suite Result
|Auto Save Interval||Specifies how often the report file is saved during execution|
|Timestamp||Specifies whether the timestamp should be computed relative to the test suite start time or relative to the test module start time or the machine time should be taken|
|Compressed Copy||Specifies whether a copy of the report should be generated as compressed folder|
|JUnit-compatible file||Specifies whether a JUnit-compatible file should be generated in addition to the test report.|
|Collect screenshots in a subdirectory||Specifies whether the created screenshots will be stored in a specific folder for each report or directly in the report folder|
|Report Template||Specifies the directory holding customized style files used instead of the Ranorex default style to present the reports; a new template can be created by pressing the button 'Create Custom Template' as explained in 'Lesson 8: Reporting - Create a Custom Report Template'; an existing template can be chosen by pressing the button 'Choose Custom Template'|
|Tracing Screenshot Mode||Specifies whether the screenshots will be captured in foreground (before action is executed) or background (while action is being executed) or capturing tracing screenshots is disabled|
|Quality||Specifies the image quality of the captured screenshots|
Test Case and Smart Folder Settings
The 'General' tab of the test case's or smart folder's properties dialog is mainly used to setup how a broken test case or smart folder impacts other test cases or smart folders within the test suite.
|Name||Name of the test case/smart folder|
|Description||Description of the test case/smart folder (same as shown within the description column of the test suite editor). Use the HTML text editor to format the description and to add links to the description. The formatted description will be represented as HTML content in the generated report|
|Report Level||Specifies the level of messages written to the report file|
|Specifies the behavior of the test case/smart folder and the test suite in case of on error. For further information please have a look at the next paragraph.|
The following figures try to illustrate several Error Behavior settings:
Continue with iteration: Ranorex will continue with the next iteration of the current test case or smart folder.
Continue with sibling: Ranorex will stop executing the current test case or smart folder and continue with the next sibling test case or smart folder instead.
Continue with parent: Ranorex will stop executing the current test case or smart folder and continue with the next parent test case or smart folder instead.
Stop: Ranorex will abort the test entirely.
Using Data in Test Suites
As already explained in Lesson 3: Data-Driven Testing you can make your automation data-driven by using variables in your modules (recordings, code modules) as well as in your repositories.
These variables can be connected to internal or external data sources and parameters.
Different kinds of Data Container
Variables are the interface between a module and the containing test case or smart folder, or - if module groups are used - the module group, respectively.
Variables can be used in
Constant values can be used in module groups to hide module variables outside a module group. This can help to reduce the complexity and increases the clarity of module groups.
For further details have a look at the section about the Module Group Editor.
In module groups, group variables are the interface between the nested modules and the nesting test case.
For further details have a look at the section about the Module Group Editor.
The columns of a data connector are called Data Columns and specify the columns from external data sources.
Have a look at the data connectors section (Data Connectors) to get an overview about the different kinds of data connectors. Data columns can be connected to variables in the data binding dialog of a test case as described in section Combining Variables with Test Data.
Module variables can also be bound to parameters. A distinction is made between local parameters, which are part of a specific test case or smart folder and are available in the test case/smart folder and all child smart folders, and global parameters which are available in the whole test suite.
Unlike local parameters, global parameters can be set from the command line as explained in section Running Tests via Command Line.
Have a look at the section Combining Variables with Parameters to see how parameters can be connected to variables.
Scope of data containers
Global parameters are available in the whole test suite. This means you can bind global parameters to any module in any test case or smart folder of a test suite.
Local parameters and data columns of a test case or smart folder are inherited by all child test cases or smart folders. This means you can bind them to
- any modules contained in the parent test case or smart folder.
- any modules contained in child test cases or smart folders.
The following figure illustrates the scope of different data containers.
Green rectangle: Scope of global parameters.
Blue rectangle: Scope of local parameters and data columns of SmartFolder2.
Red rectangle: Scope of local parameters and data columns of TestCase2.
This means that you can bind the following parameters and data columns to the modules in TestCase2:
all global parameters (green rectangle)
all local parameters and data columns of SmartFolder2 (blue rectangle)
all local parameters and data columns of TestCase2 (red rectangle)
This is illustrated in the following screenshot:
Modify Parameter and Data Column Values
Parameter values as well as data column values can be changed by a module. In order to transfer values from one module to another, parameter or data column values are updated from their bound module variables once the module has finished execution.
This means you can freely transfer values within the scope of a parameter or data column.
One example of this application would be to dynamically get a value from the UI which should be reused in another module as described in the section Lesson 5: Ranorex Recorder - Additional Editing Options - Types of Action Items addressing the Get Value action or at Code Examples - Accessing Test Case & Test Suite Context.
- Ranorex Studio - The Layout
- Lesson 1: Getting Started
- Lesson 2: Ranorex Modules - Test Actions
- Lesson 3: Data-Driven Testing
- Lesson 4: Ranorex Test Suite
- Lesson 5: Ranorex Recorder
- Lesson 6: UI Mapping with Ranorex Repository
- Lesson 7: Code Modules
- Lesson 8: Reporting
- Lesson 9: Ranorex Spy
- Ranorex Settings
- Ranorex Remote
- User Code Library
- Selenium WebDriver integration
- Code Examples
- Data Connectors
- Instrumentation Wizard
- Technology Instrumentation
- RanoreXPath Weight Rule Library
- Ranorex UI Adapter
- Mobile Testing
- Android Testing
- iOS Testing
- Web Testing
- Source Control
- Ranorex Studio IDE
- Visual Studio Integration
- System Requirements
- 64-bit Platforms
- Remotely Working with Ranorex
- Silent Installation of Ranorex
- XCOPY Deployment
- How to instructions