Enhanced Features of Image Validation

Aug 3, 2015 | Best Practices

Enhanced Features of Image Validation

This blog post will show how to use the image validation enhancements for your test automation projects introduced in Ranorex 5.4. It is now possible to log the similarity as well as the difference image of two compared images.

For image validation there are many purposes in test automation. You can for example add a test case to your recurring automated website tests checking whether the correct logo is displayed in the header area.

This example will be used to show how to work with the new feature.

The Initial Situation

For testing purposes we will compare the logo of the Ranorex website with another version of it. To make the situation more tangible, here are the two concrete examples:

This is the correct header logo:


And this is another version of the logo:


As you can see, especially the shadings of the Ranorex symbol are different. The goal is to log the exact variance of the images in the report of the test case. Additionally, screenshots with details about the areas which do not resemble should be logged, too.

Setup of the Ranorex Project

The basic structure of the test project is as follows:

Image Validation in Software Testing

In the SETUP part of the test case the browser is opened. To make sure the header image is visible, the EnsureVisible() Invoke Action is called.

In TEARDOWN the browser window used for the comparison is closed.

This is the CheckHeaderImages recording:

Enhanced Features Validation

It consists of one action: A CompareImage Validation.

In the “Screenshot Name” section the screenshot showing the differing image is chosen.

The “Repository Item” links to the repository item to validate. In this case it is the new (correct) header image.

Note: On the Ranorex website the image is actually a link. However, this is not a must-have. Every kind of repository item can be taken because a screenshot of the item is used.

Note: The size of the screenshot and the repository item need to be equal. Otherwise the validation will fail.

Enabling Enhanced Reporting Features

By default, no similarity degree is reported. This can be enabled in the properties pane. A left mouse button click on the validation action or the hot key <F4> opens the properties (by default the properties pane opens on the right-hand side of the window).

Compare Images

To log the exact similarity of the two pictures in the report, the “Report Similarity” option needs to be enabled. Setting it to “Always” means it will be reported for every test run, regardless of the result of the validation.

If the option “Report Difference Images” is enabled, images showing the actual differences in the actual and expected images will be logged. In this particular case it is changed to “OnFail” which means the images will only be reported in case of a failed validation.

Results of the Test Run

As expected, the report generated when running the test case shows a failed module because the compared images are differing.

Let’s analyze the logs step-by-step.

Firstly, there is an error message saying the validation failed. The reason is that the screenshot of the repository item did not match the specified screenshot of the differing header image.

The error message now also contains the exact similarity of the two images (within brackets). A similarity of “1”, meaning 100 percent, is expected, whereas the actual similarity only reached approximately 99.39 percent.


The next two log entries contain the two images that were compared, as before Ranorex 5.4.


The new difference images feature can be seen in the final log message.


The left image shows the difference of both images in binary notation (black-white image): Black means the compared images are exactly the same in that area; a White pixel signals that the compared images differ at that spot.

The right image visualizes the quantitative difference between the compared images. The colors are computed by subtracting the color values of the two images and applying that difference to a grey image. The greater the color difference in the compared images is, the more colored the difference image will be, whereas Grey areas signal no difference at all.

The difference images can show you, where the actual differences are located.


In this blog post you learned how to use some of the more advanced features of the Ranorex Image Validation. The basic concept of the image validation stays the same, but by changing the properties of the action a lot of different logging options can be enabled.

Reporting the similarity of two images can often be useful, e.g. for fine tuning the similarity value within an image validation.

If you have any questions, feel free to ask in the comment section.

Related Posts:

What Is Automation Testing?

What Is Automation Testing?

Test automation provides convenience and reliability to your software testing process. Learn more about how Ranorex Studio supports automated testing.

The Future of Artificial Intelligence in Test Automation

The Future of Artificial Intelligence in Test Automation

As technology continues to advance at an unprecedented rate, artificial intelligence (AI) is emerging as a transformative force across many industries. AI promises to revolutionize test automation. The end result? A more efficient, accurate, and reliable test...

Support Corner: API Testing with Ranorex Studio and a GET Request

Support Corner: API Testing with Ranorex Studio and a GET Request

Ranorex is a powerful no-code tool that automates web, mobile, and desktop application testing. In addition to the powerful no-code recording, you can utilize the Ranorex API to create code modules in C# or VB.NET. That’s beneficial because it can increase the...