black bands over captured screenshots

Mobile Testing, Android App Testing.
fsiglia
Posts: 10
Joined: Mon Sep 03, 2018 10:23 am

black bands over captured screenshots

Post by fsiglia » Mon Sep 03, 2018 4:00 pm

hello there, I have a question regarding image recognition in Ranorex. What I'm trying to accomplish is a simple algorithm which is able to compare 2 images taken from the app screen view and tell if they match. As of now my solution works, but there is a subtle problem caused by Ranorex "take screenshot" function. When I take the screenshot of an AndroidElement, the image taken is filled with black bands (the attached image talks by himself), so that 2 technically identical images are not recognized as identical because the bands pattern is different. How can I avoid this behavior? Thanks for your time!

https://drive.google.com/open?id=1myKVk ... 9sYBxfQdeJ

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3930
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: black bands over captured screenshots

Post by odklizec » Tue Sep 04, 2018 8:10 am

Hi,

At first, what's your Ranorex version? The most recent is 8.2.1. If you are using something older, try to upgrade and check if the problem still exists? I guess you will have to re instrument your app with the most recent version as well.
At next, how exactly do you take the screenshot (please post the recording/code sample or even better, sample solution)?
Could you please post a Ranorex snapshot (not screenshot) of the problematic element (and element's xpath, as stored in repo)?
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

fsiglia
Posts: 10
Joined: Mon Sep 03, 2018 10:23 am

Re: black bands over captured screenshots

Post by fsiglia » Tue Sep 04, 2018 9:23 am

I am using the latest Ranorex version available (8.2) in trial mode, as we are currently evaluating Ranorex in order to understand if it can fit into our workflow. As for the code sample, below is a snippet of my current solution:

Code: Select all


RxPath search_pattern = new RxPath(starting_element + "//");
            
IList<AndroidElement> android_elements = starting_element.Find<AndroidElement>(search_pattern , new Duration(10000) /* timeout msecs */);

IList<Bitmap> app_icons= new List<Bitmap>();
            
foreach (AndroidElement e in android_elements ) {
                
	string string_element= e.ToString();
                
        if (string_element.Contains("DragableContentViewRenderer")) { 
                    
		app_icons.Add(Imaging.CaptureImage(e));	/* this is how I capture a .png image of the element */
        } 
}

I attached the requested snapshot of the element, which has the following RxPath:

/mobileapp[@title='com.mtsbyme']/form[@title='MainActivity']/androidelement[@rid='content']/container[@containertype='Relative']/androidelement/androidelement/androidelement[1]/androidelement/androidelement/androidelement[1]/androidelement[2]/androidelement[2]//container[@containertype='Relative']/androidelement[@rid='pager']/androidelement/androidelement/androidelement[1]/androidelement[@contentdescription='{ "title": "Modo\r\nassente", "state": "NotActive"}']/androidelement[3]/?/?/androidelement

finally, here is the difference beetween the icon as viewed from the Spy, and the icon taken as screenshot:
difference.PNG
difference.PNG (20.6 KiB) Viewed 111 times
Thanks for your time and help.
Attachments
icon.rxsnp
(47.47 KiB) Downloaded 2 times

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3930
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: black bands over captured screenshots

Post by odklizec » Tue Sep 04, 2018 9:44 am

Hi,

The problem is, that the image with bands appears to be a part of the application? And the application also contains images without bands.
Android.png
Android.png (137.2 KiB) Viewed 102 times
Additionally, the xpath you are using returns 3 elements and Ranorex always (in case of multiple returned elements) picks the first one!
Android_2.png
Android_2.png (82.88 KiB) Viewed 102 times
And the code you are using is not specific enough, which Android elements exactly should be used? What images do you want to compare? Those with bands or without bands?

In case you want to take image without bands, add this platformclass at the end of xpath:
androidelement[@platformclass~'.SKCanvasView$']
or this for image with bands:
androidelement[@platformclass~'.SKCanvasViewRenderer$']
Anyway, I would suggest to improve the xpath, because the way you are using it now, is pretty fragile and most likely to fail in near future (especially all these indexes). Something like this would be much more reliable:
/mobileapp[@title='com.mtsbyme']/form[@title='MainActivity']/androidelement[@rid='content']/container[@containertype='Relative']/androidelement//container[@containertype='Relative']/androidelement[@rid='pager']//androidelement[@contentdescription~'Modo\\r\\nassente']/androidelement[@platformclass~'.DragableContentViewRenderer$']//androidelement[@platformclass~'.SKCanvasView$']
BTW, regarding the black bands, have you tried to disable (uncheck) hidden screenshot capturing?...
black-screenshots-on-test-android-t8216.html#p33242
This may not help with eliminating bands, but it's worth a try ;)
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

fsiglia
Posts: 10
Joined: Mon Sep 03, 2018 10:23 am

Re: black bands over captured screenshots

Post by fsiglia » Thu Sep 06, 2018 4:16 pm

thanks for the response, I will take a look at it as soon as I finish other activities and let you know.

fsiglia
Posts: 10
Joined: Mon Sep 03, 2018 10:23 am

Re: black bands over captured screenshots

Post by fsiglia » Mon Sep 17, 2018 2:05 pm

Here I am finally. I was able to overcome the problem by slightly changing my code. Instead of calling the Method "CaptureImage" to take a .png screenshot of the element, I call "CaptureImageHidden", which takes a clean image of the element, without black bands. Thanks for your support, was really appreciated!

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3930
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: black bands over captured screenshots

Post by odklizec » Mon Sep 17, 2018 2:09 pm

Hi,

Thanks for sharing your solution. It's funny because it's exact opposite of what I thought could be the source of your problem (hidden screenshot capturing) ;)
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration