Screenshot Validation

Technology specific object identification, supported applications, web technologies, and 3rd party controls.
SteveSM.com
Posts: 10
Joined: Tue Mar 15, 2016 11:32 am

Screenshot Validation

Post by SteveSM.com » Tue Mar 15, 2016 11:46 am

Im currently testing the Excel output from our website. Im trying to validate the spreadsheet using screenshot validation.

I currently have one test case with two recording modules. The first module selects various options before downloading and opening the spreadsheet. The second module is used to validate the contents.
I started with validating each cell, but as there are so many I tried screenshot validation.

I have two screenshot validation actions in the module, looking at two different areas of the same screenshot.
If I open the spreadsheet myself and run this module, it runs successfully. However if I run the full test case with the other module running first (downloading and opening the spreadsheet), it fails on the second screenshot validation.

It looks like its trying to compare the whole screen with the screenshot instead of the specific area I selected using the Edit Image Validation tool - but in the first validation item, ive done the same thing - except I have highlighted a different area on the screenshot.

1. Why does it work when the module is ran manually, but not as part of the full test case?

2. What is the best way to add screenshots from webpages or excel? I seem to have added the first screenshot I wanted by accident and im not sure how to add another screenshot.

SteveSM.com
Posts: 10
Joined: Tue Mar 15, 2016 11:32 am

Re: Screenshot Validation

Post by SteveSM.com » Tue Mar 15, 2016 3:54 pm

I forgot to add this to the original post, I am currently using Ranorex version 5.4.5 on Windows 7.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3937
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Screenshot Validation

Post by odklizec » Tue Mar 15, 2016 4:15 pm

Hi,

Using screenshot validation for excel files (basically anything else than simple elements/images) is way too prone to failures. You should use image validation only if there is absolutely no other way to perform proper validation. For example elements, which are not recognizable/traceable by Ranorex (typically images). And you should use it carefully and only for simple elements. Definitely not for anything like a complex table!

I would strongly encourage you to invest some time to learn how to compare whole tables against reference data stored in excel, csv, db or even Ranorex snapshots. Please take a look at the advanced samples listed here:
http://www.ranorex.com/support/user-gui ... mples.html

Using techniques from these samples will free you from manual preparation of validation steps and reference data per individual cells and you will get much more robust and error prone way to validate complex tables. You will really appreciate it in a long run!
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

SteveSM.com
Posts: 10
Joined: Tue Mar 15, 2016 11:32 am

Re: Screenshot Validation

Post by SteveSM.com » Wed Mar 16, 2016 4:17 pm

Hi odklizec,

Thanks for the response, the text based file validation appears to be the way to go if I am understanding the description correctly. It seems that I can basically compare the content of the file with another file that already exists - if this is true, then it sounds almost exactly the same as with what im trying to do, but with a file instead of an image.

Unfortunately I have no experience with coding at all so im a bit lost with the code examples.

From reading the example, it looks like its a case of just replacing filePath_Expected with the full path of the reference file, and replacing filePath_Current with the full path of the file I would be downloading - it seems a little too easy so im sure there is more to it than that, what else would I need to do?

Also in future I would like to do the same thing using PDF files, however the PDF filenames I will be downloading include a date and timestamp of when they were created - what would be the best way to compare files with variable filenames?

Thanks again.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3937
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Screenshot Validation

Post by odklizec » Thu Mar 17, 2016 8:35 am

Hi,

I'm afraid, without more details about your application under test and your ref files, it will be very hard (if not impossible) to provide you with some reasonable answers ;) I would suggest you to start with some basics. Create a sample project, where you compare simple reference file with simple table. Then if it works well, you can apply what you have learned in a real project. If you stuck somewhere, ask for help, but include a sample project with all necessary ref. files and AUT/HTML sample, etc...

You will also have to learn and understand a bit of C#. Without at least basic knowledge of C#, you will soon be stuck with problems that may be very easy to solve with basic coding skills. Most examples discussed here at forum and in the user guide often need some editing.

Honestly, the test automation is all about programming. Without at least basic programming skills, you will never unleash the full potential of Ranorex (or any other test automation tool). There are some blogs suggesting it's possible to do test automation without programming skills. And while it's partially true, person without programming knowledge will soon or later hit the wall ;) To tell you the truth, I'm not a programmer either. Just a curious guy with urge to solve problems and will to learn new stuff ;)
Last edited by odklizec on Thu Mar 17, 2016 4:12 pm, edited 1 time in total.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

SteveSM.com
Posts: 10
Joined: Tue Mar 15, 2016 11:32 am

Re: Screenshot Validation

Post by SteveSM.com » Thu Mar 17, 2016 4:01 pm

Hi Pavel,

Thanks again for the reply and the suggestion. The intended use of screenshot validation was more of a shortcut because of the amount of data in the spreadsheet that needs to be validated. Ive had a rethink and instead ive managed to condense the data I will be validating while still meeting the requirements. Because of this im happy to validate the cells that I need to individually.

Validating the spreadsheet output is a small part of what I would like to test anyway, so im happy to work around it.

Hopefully in the future when I have more time available I will be able to go back and refine each test case using code based modules.

Thanks again for your help.

Steve