Lesson 3: Data-Driven Testing

In this lesson you see how tests can be done using internal (simple data tables) or external data sets (Excel files, CSV files, SQL Databases) to do a data-driven automated test. You can use variables in recordings and even in the repositories which are connected to internal or external data sources. Additionally, you will see how a select action can be invoked for UI elements which are not visible by default in order to ensure your data-driven test case does not fail.

When you test an application it might be necessary to run the same test with different input data. Next you'll learn about:

Using Variables within Recordings

According to the KeePass application we can identify six variable input actions:

Title (WordPressDemo)
Username (admin)
Password (demo123)
URL (http://bitly.com/wp_demo)
Expires (1 Year)
IconIndex (1)

All these input actions are done within the 'AddNewEntry' recording. You need to open the recording file and identify the actions which have to be variable.

To make the input of the title variable, open the combo box as shown below and select 'As new Variable' to create a new variable.

: Create a new variable for a key sequence action item

: Create a new variable

Specify the variable name and the default value which is automatically set to the same value as it was recorded initially. If you want to alter the name or the default values, simply open the variables dialog as described in section Using Variables within the Repository.

Note: Please do not use variable names which are already in use by a recording or code module.

Repeat the previous step which makes the key sequence action used to set the value of the title field variable, and also create the variables 'varPassword', 'varUsername' and 'varURL' for the specific actions.

Using Variables within the Repository

Within the recording 'AddNewEntry' the variables 'varTitle', 'varUsername', 'varPassword' and 'varURL' are used for key sequence actions.

To make a click action data driven - for example selecting a context menu item for expires-constants like 1 week - you need to define a variable used for identification within a RanoreXPath expression.

Select that action item within the 'AddNewEntry' recording which simulates a click on a menu item and open the context menu as shown below. Click on 'Make Item Variable...' to open the RanoreXPath editor.

: 'Make Item Variable' menu item from context menu in Recorder

Typically the menu item for the 'Expires Date' in the KeePass application is identified by the accessible name attribute. Now you can define a new variable for the attribute text by clicking the button on the right.

: Create a new variable for the expires menu item

When looking at the repository, you can see that the item pointing to the menu item is now using the variable within the path expression specified previously.

: Renaming repository items

Also rename the repository item 'MenuItem1Year' to 'MI_Expires' to clarify that this repository element is not pointing to a specific menu item anymore but instead uses a variable to choose one of the menu items in a data-driven way.

Repeat the same procedure to add a new variable 'varIconIndex' into the repository item which is connected to the icon list view.

The recording 'AddNewEntry' now uses six variables. Four of them are used for key sequences directly within the recording while the others are used within the repository. To get an overview about all variables used within the recording simply click the 'Variables' button as shown below.

: Open the recording's variable dialog

: Variables currently used by the recording and repository

Creating Test Data

The sample recording now uses six variables. In this section you will learn how to connect a simple Excel table to provide the values for these variables. To create a new Ranorex data source you need to open the test suite view.

Select the test case which uses the 'AddNewEntry' recording and open the 'Data Source' dialog by clicking the 'Manage Data Sources' button in the tool bar.

: Open the 'Manage data sources ...' dialog

Assuming someone is keeping his/her passwords in an excel-file, the following example could be helpful in more safely managing the information.

This Excel file contains the fields Title, Username, Password, URL, Expires and IconIndex.

Note: The top row is handled as a header. Therefore the cells of the first row will be interpreted as captions for the current column.

: Use an Excel file for data-driven testing

Now create a new data connector by clicking on 'Use Excel file...'.

: Create a new Excel connector

: Specify the filename from the Excel file

For further information about different Ranorex data connector types have a look at Data Connectors.

Combining Variables with Test Data

In order to combine your Excel file with the current test case - and finally with the 'AddNewEntry' recording which is used by the test case - you need to open the data binding dialog using the context menu as shown below.

: Open data binding dialog

Note: Please assure that the 'ExcelConnector' is chosen in the 'Data Source' tab sheet (drop down in the middle of the dialog).

Now you can connect each column specified within the data table with variables used by the test case. You can also map multiple variables to one data column. You currently only have variables specified within the 'AddNewEntry' module.

: Connect data columns with module variables

Combining Variables with Parameters

Parameters can be useful when automated tests are executed under different environment-related conditions. An automated test on Windows 7, for example, may need different path values in comparison to a test execution on Windows XP. Parameters can be created in Ranorex Studio in order to be connected with variables.

In addition to typical test data specified within data tables as shown before, a test suite and its test cases allow you to declare global and local parameters. These parameters can also be bound to module variables. This type of data driven execution can help with environment-related attribute values and similar issues as the following example describes.

The test case starts with a recording module called 'StartSUT'. The recording only contains a single action item used to start the KeePass application. Open the recording and create a new variable called 'varApplicationPath'.

: Create a new variable for the execution path of system under test

The execution path of the system under test has to be declared as a global parameter on the test suite level in order to combine it with multiple test cases of the test suite. Use the context menu within the test suite view and open the parameters dialog as shown below.

: Open global parameters dialog

: Define a new global parameter for the test suite

Specify a new parameter name by clicking in the first cell of the column 'Name'. Now specify a value for the new parameter. Just copy and paste the execution path from the default value shown within the variables dialog of the recording 'StartSUT'. Click 'OK' to apply the changes and to close the dialog. Read more about how to open a recording's variable dialog in Lesson 5: Ranorex Recorder - Recorder Variables.

Now you are ready to combine the global parameter with the variable created within the recording before. Open the test case's data binding dialog the same way you did previously for combining variables with the data table.

: Bind global parameter to the variable used within recording 'StartSystemUnderTest'

Before you execute the data driven test case, you have to make sure that the values specified within the Excel file can be used correctly within the 'AddNewEntry' recording. The values for the title, username, password, URL and expiration will not cause any problems during automation. Considering a more advanced situation such as selecting an item from a list view which is not visible by default; there will be a problem because of the current visibility state of the item. This situation will be handled in the next section.

Invoking Actions: Selection

KeePass offers a list of icons which can be accessed via the button next to the label 'Icon':

: Button to open the Icon Picker

: Form for choosing an icon for a password entry

Icons with an index of 0 to 49 are visible without using the scroll bar so icons with indexes lower than 49 could even be accessed easily when using a data-driven approach. In order to select an icon with a higher index (e.g. 66 for the US dollar icon) in an automated way, a simple click on an invisible item would fail. It is recommended to add an 'Invoke Action' to select the icon regardless of the visibility of the item in the listview.

First of all an additional action should be added to the recording right before the action representing the click on the icon.

Select one action before the click action (#5) and click the 'Add New Action' button as shown below.

: Adding a new 'Invoke Action' to the recording

After adding the new item you need to specify for which repository element the invoke action should be used. Simply open the 'Select Repository Item' dialog as shown below and select 'IconItem'.

: Open the 'Select Repository Item' dialog by clicking on the highlighted button and choosing 'ListItem_Icon' from the repository

After assigning the repository item to the invoke action, the recorder presents the methods suitable for the role list item. The role 'ListItem' shows a 'Select' method which you can chose as seen in following figure.

: Choose 'Select' method

Also delete the initially recorded mouse click action (action item #7 shown in previous figure) because now the recording uses the new invoke action to select the list item instead of the click action.

Executing Data Driven Tests

Note: Before executing the test suite, it's necessary to modify the recording 'ValidateEntry' in order to be independent from the constant values used during recording. Therefore the constant value 'WordPressDemo' should be replaced by variables (match-value in validation action and text-attribute-value in RanoreXPath for the assigned repository item) as you learned in the previous sections. Don't forget to bind the variables to the excel-column.

Now switch back to the test suite view and start your data driven test by clicking the 'Run' button.

: Report file showing two iterations of the test case

After executing the test suite the report file shows the results for each iteration. The summary chart counts each iteration as a single test case run.