Referring to software testing – validation is the verification of correctness with respect to behavior and/or result(s) of the tested software. The test process intends to prove that a given user interface scenario leads to a defined software behavior and/or result. The different approaches to test validation with Ranorex Studio are introduced and explained herein.
In this chapter
Simple validation examples
At Ranorex, we like to keep it smart and easy for users! So, let’s focus on two introductory examples of test validation at first! This shall give you a feeling of what test validation is and how important test validation is.
Imagine a standard pocket calculator you use to calculate two expenses:
Important to know:
- The expected result sums up to 57.79.
- You may want to ask – OK? And what’s this all about?
- With respect to test validation, the answer is simple. The manufacturer of the pocket calculator needs to guarantee that not only THIS result is correct, but also all other results. From the simplest addition to any other complex built-in mathematical formula.
This is where test validation comes into play. By means of automated testing, the pocket calculator manufacturer can cover as many different calculations with different input data and results as necessary in order to prove that the calculator works correctly.
Fly-by-wire (FBW) is a system that replaces the conventional manual flight controls of an aircraft with an electronic interface. The movements of flight controls are converted to electronic signals transmitted by wires (hence the fly-by-wire term), and flight control computers determine how to move the actuators at each control surface to provide the ordered response. The fly-by-wire system also allows automatic signals sent by the aircraft’s computers to perform functions without the pilot’s input, as in systems that automatically help stabilize the aircraft, or prevent unsafe operation of the aircraft outside of its performance envelope.
We are pretty sure that you agree that these computer systems always and without a single exception should behave and function as specified and that any user input from the pilot must lead to the specified result.
Test validation is crucial! Not only with simple computer software like calculators, or complex flight control computers in aircraft, but in every software system in between!
Before starting with various topics in test validation let us finally focus on a small, but important issue. You need to be aware that we are coping with ‘indirect test validation’ only!
Let us explain what this means!
Remember the introductory example of the calculator. Test validation means, that we verify if the user input sequence of:
Calculator calculation steps
leads to the result of:
Windows calculator result display
Important to know:
It is important to know, that we only verify that the displayed result is similar to the expected result based on the user input! We are not able to verify if the calculator effectively works correctly!
- There might be many operating and processing steps between the user input and the display of the result on the screen.
- We are only able to match the displayed result against an initial sequence of user inputs actions and determine if the result or behavior is as expected.
In other words, if test validation fails, there is no evidence of what went wrong. It could be that the software processes everything correctly, but the displaying of the result went somehow wrong. Test data may contain erroneous data. You always need to be aware that we are not able to validate the internal processing steps within a computer system, but only the ‘outside’ visible results.
Keep that in mind when designing your test cases!