This blog post will show how to use the image validation enhancements for your test automation projects introduced in Ranorex 5.4. It is now possible to log the similarity as well as the difference image of two compared images.
For image validation there are many purposes in test automation. You can for example add a test case to your recurring automated website tests checking whether the correct logo is displayed in the header area.
This example will be used to show how to work with the new feature.
- The Initial Situation
- Setup of the Ranorex Project
- Enabling Enhanced Reporting Features
- Results of the Test Run
The Initial Situation
This is the correct header logo:
And this is another version of the logo:
As you can see, especially the shadings of the Ranorex symbol are different. The goal is to log the exact variance of the images in the report of the test case. Additionally, screenshots with details about the areas which do not resemble should be logged, too.
Setup of the Ranorex Project
In the SETUP part of the test case the browser is opened. To make sure the header image is visible, the EnsureVisible() Invoke Action is called.
In TEARDOWN the browser window used for the comparison is closed.
This is the CheckHeaderImages recording:
It consists of one action: A CompareImage Validation.
In the “Screenshot Name” section the screenshot showing the differing image is chosen.
The “Repository Item” links to the repository item to validate. In this case it is the new (correct) header image.
Note: On the Ranorex website the image is actually a link. However, this is not a must-have. Every kind of repository item can be taken because a screenshot of the item is used.
Note: The size of the screenshot and the repository item need to be equal. Otherwise the validation will fail.
Enabling Enhanced Reporting Features
By default, no similarity degree is reported. This can be enabled in the properties pane. A left mouse button click on the validation action or the hot key <F4> opens the properties (by default the properties pane opens on the right-hand side of the window).
To log the exact similarity of the two pictures in the report, the “Report Similarity” option needs to be enabled. Setting it to “Always” means it will be reported for every test run, regardless of the result of the validation.
If the option “Report Difference Images” is enabled, images showing the actual differences in the actual and expected images will be logged. In this particular case it is changed to “OnFail” which means the images will only be reported in case of a failed validation.
Results of the Test Run
Let’s analyze the logs step-by-step.
Firstly, there is an error message saying the validation failed. The reason is that the screenshot of the repository item did not match the specified screenshot of the differing header image.
The error message now also contains the exact similarity of the two images (within brackets). A similarity of “1”, meaning 100 percent, is expected, whereas the actual similarity only reached approximately 99.39 percent.
The next two log entries contain the two images that were compared, as before Ranorex 5.4.
The new difference images feature can be seen in the final log message.
The left image shows the difference of both images in binary notation (black-white image): Black means the compared images are exactly the same in that area; a White pixel signals that the compared images differ at that spot.
The right image visualizes the quantitative difference between the compared images. The colors are computed by subtracting the color values of the two images and applying that difference to a grey image. The greater the color difference in the compared images is, the more colored the difference image will be, whereas Grey areas signal no difference at all.
The difference images can show you, where the actual differences are located.
In this blog post you learned how to use some of the more advanced features of the Ranorex Image Validation. The basic concept of the image validation stays the same, but by changing the properties of the action a lot of different logging options can be enabled.
Reporting the similarity of two images can often be useful, e.g. for fine tuning the similarity value within an image validation.
If you have any questions, feel free to ask in the comment section.