Image validation not working properly

Ask general questions here.
User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3921
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Image validation not working properly

Post by odklizec » Thu Feb 16, 2017 8:08 am

Hi,

Of course, you don't have to re-record image validations with newer Ranorex version. Unfortunately, without seeing your app and the test suite (or at least related recording with repo), it's pretty hard to tell what's wrong. Image validation is generally somewhat quirky way to automate and validate things. But to be honest, Ranorex image validation works quite well for me. I have one test, which is nearly 100% image-based and it's pretty reliable. The test runs 24/7 in 15m intervals and it fails only rarely, usually, due to some network issues, not because of image validation.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

rastek
Posts: 157
Joined: Wed Aug 06, 2014 12:00 pm

Re: Image validation not working properly

Post by rastek » Thu Feb 16, 2017 9:23 am

when I run from suite or begining of recording it fails but when I run only validation action it pass.

So I assume it is a timing issue and put 5 seconds delay which is very enough but again failing in the suite !

Something really buggy, it is not realted to time delay, it is failing during overall running.

It always expected = 1

I tried something, run only validation action against a different page and value was similarity similarity actual='0.984852062487312', expected='1


That means normally my 80 % similarity is not enought for validation even if it uses real value I want.

and still do not understand that where that 1 comes from ?

rastek
Posts: 157
Joined: Wed Aug 06, 2014 12:00 pm

Re: Image validation not working properly

Post by rastek » Thu Feb 16, 2017 5:22 pm

Eventually I rerecorded problematic screens again and started to work, but still do not understand that expected=1 issue..

rastek
Posts: 157
Joined: Wed Aug 06, 2014 12:00 pm

Re: Image validation not working properly

Post by rastek » Fri Feb 17, 2017 2:23 pm

Please check attached png files for the faulty result.

As you can see from images have tested subimage saying "Step 29: Load Distribution" against a screen which says "Step 31: Install Oracle" and get pass with 99% similarity, how you explain this ??

here are my details.
I made Similarity value again 1 and run against a false screen, and this time it bring my old value 75% in result and again 0.99 percent similarity for a totally different piece (compare method when another screen is seen and I see that different screen even in report screenshot but it passes :)

Screenshot of item 'asplatformRepository.GENBANDApplicationServerInstallWizar.GENBANDApplicationServerInstallWizar' does contain the specified image (similarity actual='0.993177456167652', expected='0.75').


never takes real Similarity value, this image validation is totally headache or I miss something here ?? It is like a joke.
Attachments
fromReport.png
fromReport.png (71.9 KiB) Viewed 479 times
fromValidateAction.png
fromValidateAction.png (28.74 KiB) Viewed 479 times

rastek
Posts: 157
Joined: Wed Aug 06, 2014 12:00 pm

Re: Image validation not working properly

Post by rastek » Fri Feb 17, 2017 5:12 pm

Odklizec, I do not know who is responsible for bugs but there is definitely bug in Similarity value. It does not update itself until a restart, I also mentioned the problem in Ideas section of the forum.

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Image validation not working properly

Post by Support Team » Tue Feb 21, 2017 4:11 pm

Hi everybody,

rastek is already in contact with Ranorex Support to discuss those issues by email.

Sincerely,
Johannes
.
Image