Possible to click on AUT based on image in local drive

Ask general questions here.
Posts: 22
Joined: Tue Jul 18, 2017 10:59 am

Possible to click on AUT based on image in local drive

Post by Sai » Tue Aug 08, 2017 2:11 pm

Hello Users,

Happy to post here again!

I have thousands of images/screenshots in my local drive (system) to validate against AUT during run time.

I am able to store local images into variables and compare against screenshots of repository items

Bitmap bmp1 = Ranorex.Imaging.Load(@"..\..\images\Selenium_Integration.jpg");

but i need to perform user actions like click etc on AUT by using variables, not with help of repository. Is it possible to perform user actions like below by using variables


Any reply appreciable.


Posts: 297
Joined: Thu Feb 23, 2017 10:37 am
Location: Benedikt, Slovenia

Re: Possible to click on AUT based on image in local drive

Post by McTurtle » Wed Aug 09, 2017 6:37 pm

Hello Sai,

I was playing around a bit... It is possible to do what you would like to achieve. I have built an example based on the Windows 10 calculator. In order to implement something similar you need at least the RanoreXPath to the AUT so that you can create one element. The code that I have used is below:
public void FindBMP()
        	//Crate bitmap from 7-button
        	Bitmap bitmap=new Bitmap("7.bmp");
        	//Create an unknown from the AUT
        	Unknown AUT=Element.FromPath("/winapp[@packagename='Microsoft.WindowsCalculator']");
        	//Create some basic imaging options so that we can play with similarity
        	Imaging.FindOptions options=new Imaging.FindOptions();
        	//Create a list of all matches of your bitmap in the aut
 			IList <Imaging.Match> matches=Imaging.Find(AUT,bitmap, options);
 			//Sort the list by similarity, you need "using System.Linq;" for this
 			IEnumerable<Imaging.Match> _sorted_matches=matches.OrderBy(f=>f.Similarity);
 			//Create a list from the IEnumerable again
 			IList<Imaging.Match> sorted_matches=_sorted_matches.ToList();
 			//Click on the coordinates of the last element of the sorted list
I have attached the sample solution to this post. You need to start the Windows 10 calculator and then you can run the solution. It should click the button 7. If you find out by debugging that the list 'matches' does not get populated then you need to create your own bmp of the 7-button or lower the similarity.

Please let me know if this solution is useful to you.

(41.69 KiB) Downloaded 44 times