Lesson 3 - Data-driven Testing
In this lesson you see how tests can be done using internal (simple data tables) or external data sets (Excel files, CSV files, SQL Databases) to do a data-driven automated test. You can use variables in recordings and even in the repositories which are connected to internal or external data sources. Additionally, you will see how a select action can be invoked for UI elements which are not visible by default in order to ensure your data-driven test case does not fail.
When you test an application it might be necessary to run the same test with different input data. Next you'll learn about:
Using Variables within Recordings
According to the KeePass application we can identify six variable input actions:
- Title (WordPressDemo)
- Username (admin)
- Password (demo123)
- URL (http://bitly.com/wp_demo)
- Expires (1 Year)
- IconIndex (1)
All these input actions are done within the 'AddEntry' recording. You need to open the recording file and identify the actions which have to be variable.
To make the input of the title variable, open the combo box as shown below and select 'As new Variable' to create a new variable.
Create a new variable for a key sequence action item
Create a new variable
Specify the variable name and the default value which is automatically set to the same value as it was recorded initially. If you want to alter the name or the default values, simply open the variables dialog as described in section.
Note Please do not use variable names which are already in use by a recording or code module.
Repeat the previous step which makes the key sequence action used to set the value of the title field variable, and also create the variables 'varPassword', 'varUsername' and 'varURL' for the specific actions.
Using Variables within the Repository
Within the recording 'AddEntry' the variables 'varTitle', 'varUsername', 'varPassword' and 'varURL' are used for key sequence actions.
To make a click action data driven - for example selecting a context menu item for 'expires' value (1 week, 1 year...) - you need to define a variable used for identification within a RanoreXPath expression.
Select that action item within the 'AddEntry' recording which simulates a click on a menu item and open the context menu as shown below. Click on 'Make Item Variable...' to open the path editor.
'Make Item Variable' menu item from context menu in Recorder
Create a new variable for the expires menu item
When looking at the repository, you can see that the item pointing to the menu item is now using the variable within the path expression specified previously.
Variable repository item
Repeat the same procedure to add a new variable 'varIconIndex' into the repository item which is connected to the icon list view.
The recording 'AddEntry' now uses six variables. Four of them are used for key sequences directly within the recording while the others are used within the repository. To get an overview of all variables used within the recording simply click the 'Variables' button as shown below.
Open the recording's variable Dialog
Variables currently used by the recording and repository
Creating Test Data
Select the test case which uses the 'AddEntry' recording and open the 'Data Source' dialog by clicking the 'Manage Data Sources' button in the tool bar.
Open the 'Manage data sources ...' dialog
This Excel file contains the fields Title, Username, Password, URL, Expires and IconIndex.
Note The top row is handled as a header. Therefore the cells of the first row will be interpreted as captions for the current column.
Use an Excel file for data-driven testing
Create a new Excel connector
Specify the filename from the Excel file
Note To protect proprietary data like passwords you can mask specific columns of your data source.
Note It is recommended to store your Excel data files in the native binary format xlsb instead of the suggested default format xlsx. The binary format is up to ten times faster than xlsx, and supported since Microsoft Office 2007.
Combining Variables with Test Data
Open data binding dialog
Note Please assure that the 'ExcelConnector' is chosen in the 'Data Source' tab sheet (drop down in the middle of the dialog).
Now you can connect each column specified within the data table with variables used by the test case. You can also map multiple variables to one data column. You currently only have variables specified within the 'AddEntry' module.
Connect data columns with module variables
Note Variable binding (e.g. to a data column) only takes place if the test automation is run from the test suite view. When running modules separately (e.g. "Play" in recorder), variable's default values are used.
Combining Variables with Parameters
Parameters can be useful when automated tests are executed under different environment-related conditions. An automated test on Windows 7, for example, may need different path values in comparison to a test execution on Windows XP. Parameters can be created in Ranorex Studio in order to be connected with variables.
In addition to typical test data specified within data tables as shown before, a test suite and its test cases allow you to declare global and local parameters. These parameters can also be bound to module variables. This type of data driven execution can help with environment-related attribute values and similar issues as the following example describes.
The test case starts with a recording module called 'StartSUT'. The recording only contains a single action item used to start the KeePass application. Open the recording and create a new variable called 'varApplicationPath'.
Create a new variable for the execution path of system under test
The execution path of the system under test has to be declared as a global parameter on the test suite level in order to combine it with multiple test cases of the test suite. Use the context menu within the test suite view and open the parameters dialog as shown below.
Open global parameters dialog
Define a new global parameter for the test suite
Specify a new parameter name by clicking in the first cell of the column 'Name'. Now specify a value for the new parameter. Just copy and paste the execution path from the default value shown within the variables dialog of the recording 'StartSUT'. Click 'OK' to apply the changes and to close the dialog. Read more about how to open a recording's variable dialog in Lesson 5 - Ranorex Recorder - Recorder Variables.
Now you are ready to combine the global parameter with the variable created within the recording before. Open the test case's data binding dialog the same way you did previously for combining variables with the data table.
Bind global parameter to the variable used within recording 'StartSystemUnderTest'
Note By pressing the button 'Auto-Create' a parameter for each unbound module variable will be generated based on the name of the module variable. By pressing the button 'Auto-Bind' unbound module variables will be bound to parameters having identical names.
Note When moving the mouse over a module in the test suite a tool tip will appear displaying which variables are bound and which are not
Before you execute the data driven test case, you have to make sure that the values specified within the Excel file can be used correctly within the 'AddEntry' recording. The values for the title, username, password, URL and expiration will not cause any problems during automation. Considering a more advanced situation such as selecting an item from a list view which is not visible by default; there will be a problem because of the current visibility state of the item. This situation will be handled in the next section.
Note Variable binding (e.g. to a global parameter) only takes place if the test automation is run from the test suite view. When running modules separately (e.g. "Play" in recorder), variable's default values are used.
Button to open the Icon Picker
Form for choosing an icon for a password entry
Icons with an index of 0 to 49 are visible without using the scroll bar so icons with indexes lower than 49 could even be accessed easily when using a data-driven approach. In order to select an icon with a higher index (e.g. 66 for the US dollar icon) in an automated way, a simple click on an invisible item would fail. It is recommended to add an 'Invoke Action' to select the icon regardless of the visibility of the item in the listview.
First of all an additional action should be added to the recording right before the action representing the click on the icon.
Select one action before the click action (#5) and click the 'Add New Action' button as shown below.
Adding a new 'Invoke Action' to the recording
Open the 'Select Repository Item' dialog by clicking on the highlighted button and choosing 'ListItem_Icon' from the repository
Choose 'Select' method
Executing Data Driven Tests
Before executing the test suite, it's necessary to modify the recording 'ValidateEntry' in order to be independent from the constant values used during recording. Therefore the constant value 'WordPressDemo' should be replaced by variables. First, please replace the match-value in the validation action.
Replace match value by a new variable
Finally, please modify the text-attribute-value in the RanoreXPath expression of the assigned repository item as learned in the previous sections.
Replace value by variable in RanoreXPath
Note Don't forget to bind the variables to the excel-column.
Now switch back to the test suite view and start your data driven test by clicking the 'Run' button.
Report file showing two iterations of the test case