Enhanced Features of Image Validation

Aug 3, 2015 | Best Practices

Enhanced Features of Image Validation

This blog post will show how to use the image validation enhancements for your test automation projects introduced in Ranorex 5.4. It is now possible to log the similarity as well as the difference image of two compared images.

For image validation there are many purposes in test automation. You can for example add a test case to your recurring automated website tests checking whether the correct logo is displayed in the header area.

This example will be used to show how to work with the new feature.

The Initial Situation


For testing purposes we will compare the logo of the Ranorex website with another version of it. To make the situation more tangible, here are the two concrete examples:

This is the correct header logo:

Enhanced_Features-Correct_Logo

And this is another version of the logo:

Enhanced_Features-Wrong_Logo

As you can see, especially the shadings of the Ranorex symbol are different. The goal is to log the exact variance of the images in the report of the test case. Additionally, screenshots with details about the areas which do not resemble should be logged, too.

Setup of the Ranorex Project


The basic structure of the test project is as follows:

Image Validation in Software Testing

In the SETUP part of the test case the browser is opened. To make sure the header image is visible, the EnsureVisible() Invoke Action is called.

In TEARDOWN the browser window used for the comparison is closed.

This is the CheckHeaderImages recording:

Enhanced Features Validation

It consists of one action: A CompareImage Validation.

In the “Screenshot Name” section the screenshot showing the differing image is chosen.

The “Repository Item” links to the repository item to validate. In this case it is the new (correct) header image.

Note: On the Ranorex website the image is actually a link. However, this is not a must-have. Every kind of repository item can be taken because a screenshot of the item is used.

Note: The size of the screenshot and the repository item need to be equal. Otherwise the validation will fail.

Enabling Enhanced Reporting Features


By default, no similarity degree is reported. This can be enabled in the properties pane. A left mouse button click on the validation action or the hot key <F4> opens the properties (by default the properties pane opens on the right-hand side of the window).

Compare Images

To log the exact similarity of the two pictures in the report, the “Report Similarity” option needs to be enabled. Setting it to “Always” means it will be reported for every test run, regardless of the result of the validation.

If the option “Report Difference Images” is enabled, images showing the actual differences in the actual and expected images will be logged. In this particular case it is changed to “OnFail” which means the images will only be reported in case of a failed validation.

Results of the Test Run


As expected, the report generated when running the test case shows a failed module because the compared images are differing.

Let’s analyze the logs step-by-step.

Firstly, there is an error message saying the validation failed. The reason is that the screenshot of the repository item did not match the specified screenshot of the differing header image.

The error message now also contains the exact similarity of the two images (within brackets). A similarity of “1”, meaning 100 percent, is expected, whereas the actual similarity only reached approximately 99.39 percent.

Enhanced_Features-Report1

The next two log entries contain the two images that were compared, as before Ranorex 5.4.

Enhanced_Features-Report2

The new difference images feature can be seen in the final log message.

Enhanced_Features-Report3

The left image shows the difference of both images in binary notation (black-white image): Black means the compared images are exactly the same in that area; a White pixel signals that the compared images differ at that spot.

The right image visualizes the quantitative difference between the compared images. The colors are computed by subtracting the color values of the two images and applying that difference to a grey image. The greater the color difference in the compared images is, the more colored the difference image will be, whereas Grey areas signal no difference at all.

The difference images can show you, where the actual differences are located.

Conclusion


In this blog post you learned how to use some of the more advanced features of the Ranorex Image Validation. The basic concept of the image validation stays the same, but by changing the properties of the action a lot of different logging options can be enabled.

Reporting the similarity of two images can often be useful, e.g. for fine tuning the similarity value within an image validation.

If you have any questions, feel free to ask in the comment section.

Related Posts:

Effective Black Box Testing Methods You Need to Try

Effective Black Box Testing Methods You Need to Try

When users open software solutions, they expect them to function as needed. For example, when a business analyst opens Excel, they hope to work with data without requiring knowledge of what’s happening with the application internally. If something breaks, they won’t...

8 Steps to Create a Data Migration Plan

8 Steps to Create a Data Migration Plan

When companies change systems or move information to a more secure location, they typically need to perform a data migration. If a company wants to use cloud-based solutions, it must transfer existing information from localized hardware to a cloud environment. A...