Introduction to test validation | Ranorex Studio User Guide
Help CenterUser GuideRanorex Studio fundamentals

Test validation

In software testing, validation is the process of verifying that the behavior and/or responses produced by the application under test match the expected results. This chapter describes the test validation approaches available in Ranorex Studio.

For simplicity, this chapter demonstrates validations that use constant values to determine whether the application returns an expected result. In practice, you will normally want to use variables for validation rather than constant values.

Reference

To learn how to use variables in validation actions, refer to the following section:

Ranorex Studio advanced > Variables and parameters > ⇢ Validation variables.

In this chapter

    Understanding validation

    Let’s begin with a simple example of test validation. Imagine a standard pocket calculator that you use to add two numbers:

    Example calculation

    Example calculation

    Test validation succeeds if the calculator returns the expected result, 57.79.

    However, the manufacturer of the pocket calculator needs to guarantee that not only this result is correct, but also all other results, from the simplest addition to any other complex built-in mathematical formula.

    This is where test validation comes into play. Automated testing makes it possible to cover as many different calculations with different input data and results as necessary in order to prove that the calculator works correctly.

    Indirect validation

    It is important to understand that GUI test automation provides indirect test validation only. This means that we verify whether the user input sequence of:

     

    Calculator calculation steps
    leads to the result of:
    Windows calculator result display
    Indirect validation verifies that the displayed result is similar to the expected result based on the user input. It does not ensure whether the calculator works correctly.

    • There might be many operating and processing steps between the user input and the display of the result on the screen.
    • We are only able to match the displayed result against an initial sequence of user inputs actions and determine if the result or behavior is as expected.

    In other words, if test validation fails, there is no evidence of what went wrong. It could be that the software processes everything correctly, but the displaying of the result went somehow wrong. Test data may contain erroneous data. You always need to be aware that we are not able to validate the internal processing steps within a computer system, but only the ‘outside’ visible results. Keep that in mind when designing your test cases.

    Validation Process

    Test validation is a process with four consecutive steps which are listed below. You can create a test validation action during recording or add it manually later.

    This is the caption of the image

    Activate validation

    Within Ranorex, validation is treated as a special type of action. So, to “activate” a test validation simply means to add a validation action to the current recording module. The instructions below describe adding a test validation during a recording session. As mentioned earlier, validation actions can also be added manually after recording.

    Reference

    If you are not familiar with test actions, we recommend reading

    Ranorex Studio fundamentals ⇢ Actions

    Activating validation

    To configure text-based validation:

    • Click Validate to activate text-based validation mode.

    To configure image-based validation:

    • Activate the switch Image based recording.
    • Click Validate to activate image-based validation mode.
    Note icon

    Note

    The recording session pauses automatically to allow you to insert the test validation action. This is indicated by the purple bar in the Recorder control center.

    Select the validation element

    The second step of the validation process is to select the UI element to be validated. The selection process is independent of the type of validation (i.e. text/attribute/image-based validation).

    Move the mouse over the UI element to be validated. Look for a purple frame to appear around the UI element, indicating that Ranorex Studio has identified it. Click to select the UI element.

    Selection of a text-based validation element.
    Selection of an image-based validation element.
    tipp icon

    Hint

    • Identifying the correct UI element can be challenging, especially when it is nested closely with other UI elements.
    • If you select the wrong UI element by mistake, you can easily change it during the next step.

    Confirm the validation element

    The third step of the validation process is to confirm that the correct UI element was selected for validation. If so, click Next. Otherwise, correct the selection.

    Confirmation dialog components

    Components of validation confirmation dialog

    UI element location within GUI

    • Ranorex identifies every UI element within the application under test.
    • All UI elements are managed in a hierarchical structure.
    • The names of the UI elements are derived from their underlying type and content.

    UI element states and attributes

    • Each UI element is defined by a set of attributes and states.
    • An example of a state is Enabled with a current value of True.
    • An example of an attribute is the UI element’s name. In the example above, the text attribute has the current value lblWelcomeMessage.

    Screenshot of the UI element

    • A screenshot of the selected UI element is displayed for confirmation.

    Reference

    UI elements, their representation within Ranorex and their states and attributes are covered in

    Ranorex Studio advanced > ⇢ UI elements

    Confirmation action

    Click Next to confirm validation element

    Confirmation of text-based validation element.

    Confirmation of image-based validation element.

    Define validation attributes

    Each UI element is defined by states and attributes. During the final step, specify the particular attribute or state to be used for validation, as shown below.

    Attribute/sub-image selection for validation UI elements

    Attribute/state selection for text/attribute-based validation

    • Any of the available attributes/states can be selected.
    • The text attribute is the default attribute for text-based validation.
    • Select the attribute and click OK to confirm.

    (Sub-)image selection for imagebased validation

    • A sub-region of the image can be selected by drawing a pink rectangle within the image.
    • If you do not select a sub-region, the complete image is used for test validation.

    Image validation mode

    • None = No image-based validation; validation discarded.
    • Contains = Checks that a matched image is contained within a reference validation image.
    • Compare = Checks a matched image against a complete reference validation image.

    Validation examples

    To learn more about the three types of validation, refer to the examples linked below:

    Further reading

    To learn more about UI elements, refer to

    Ranorex Studio advanced > ⇢ UI elements.

    To learn more about image-based automation, refer to

    Ranorex Studio advanced > ⇢ Image-based automation.