Android Touch & Image Recognition

Class library usage, coding and language questions.
WMacNeil
Posts: 5
Joined: Tue Jun 11, 2013 2:32 pm

Android Touch & Image Recognition

Post by WMacNeil » Fri Jun 14, 2013 5:45 pm

Is it possible in Android to perform a click/touch operation based on screen coordinates?
Also, is it possible to perform image recognition on Android?

WMacNeil
Posts: 5
Joined: Tue Jun 11, 2013 2:32 pm

Re: Android Touch & Image Recognition

Post by WMacNeil » Fri Jun 14, 2013 8:00 pm

Hey Team. Your help would be appreciated. This is on an android app I'm attempting to automate.

Code: Select all

      Bitmap bmp = Imaging.CaptureImage(repo.NetSotiMobicontrol.Cancel);
      bmp.Save(@"C:\Users\WMacNeil\Documents\myImage.bmp"); 
      Bitmap bmp2 = Imaging.Load(@"C:\Users\WMacNeil\Documents\myImage.bmp");

      if (Ranorex.Imaging.Contains(repo.NetSotiMobicontrol.Cancel,bmp2) == true)  
      {  
           Report.Success("Image found within application");  
      } 
      else{
           Report.Failure("Image not found");  
      }
The above code works. But lets say I had an image of a button without capturing it initally which closely resembled the button I was looking to match. Also, lets say the element I was looking to match didn't exist in the repository.

This is the hypothetical code, which does not work. Image can't be found. Works in Gui or Web based applications only.

Code: Select all

  
      //this is the image which closely resembles the match I'm looking for
      Bitmap bmp2 = Imaging.Load(@"C:\Users\WMacNeil\Documents\myImage2.bmp");

      //I am not aware of the element which matches my image loaded above
      if (Ranorex.Imaging.Contains(repo.NetSotiMobicontrol.Self,bmp2) == true)  
      {  
           Report.Success("Image found within application");  
           repo.NetSotiMobicontrol.Self.Touch(bmp2)
      } 
      else{
           Report.Failure("Image not found");  
      }

MarkusE
Posts: 2
Joined: Mon Jun 17, 2013 2:45 pm

Re: Android Touch & Image Recognition

Post by MarkusE » Mon Jun 17, 2013 4:11 pm

Hi,

If the image change a little you have to set the similarity property to a lower value in order to identify it and in order to click/touch on the position of the found element/image you could use the following code:
int timeOut = 10000;
Imaging.FindOptions imgFindOptions = new Imaging.FindOptions();
imgFindOptions.Timeout = timeOut;
imgFindOptions.Similarity = 0.8; // could also be set to a lower or higher value
Bitmap bmp2 = Imaging.Load(@"C:\Users\WMacNeil\Documents\myImage2.bmp");
        	
// runtime validation by comparing against the repository object which is the entire app
Report.Info("Trying to find image for " + timeOut + " milliseconds.");
Ranorex.Imaging.Contains(repo.NetSotiMobicontrol.Self, bmp2, imgFindOptions);
        	
// find out relative location of the specific image
Ranorex.Imaging.Match match = Imaging.FindSingle(repo.NetSotiMobicontrol.Self, bmp2, imgFindOptions);
Report.Info("Location of the image (relatively to the element): " + match.Location);
        	
repo.NetSotiMobicontrol.Self.Touch(new Location((int)(match.Location.X + match.Size.Width/2), (int)(match.Location.X +match.Location.Y/2)));
I hope this will help you to solve the issue!

Regards,
Markus

WMacNeil
Posts: 5
Joined: Tue Jun 11, 2013 2:32 pm

Re: Android Touch & Image Recognition

Post by WMacNeil » Tue Jun 18, 2013 12:52 am

Thanks Markus. I didn't realize the Contains method had an overloaded method. Should have checked the API. Thanks for your help. I will be sure to put this to the test.