Mouse click on wrong place with Edge

Technology specific object identification, supported applications, web technologies, and 3rd party controls.
TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Mouse click on wrong place with Edge

Post by TimoL » Mon Mar 16, 2020 3:27 pm

I run the same test with different browsers. Other browsers works fine, but with Edge a mouse click is performed outside the object, which has this kind of path: .//a[@innertext='xxx' and @enabled='true'] . Both Ranorex Studio and Spy highlights the correct element (and only that one) and if the action is run separately, it works fine. But when the whole test is run, the click is performed outside the element. What could be the reason?

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6180
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Mouse click on wrong place with Edge

Post by odklizec » Mon Mar 16, 2020 3:32 pm

Hi,

At first, which version of Edge (chromium-based or older) and Ranorex version are you using? Please remember that Chromium-based Edge is officially supported from 9.3.0! If you are using new Edge, you should use also new Ranorex 9.3.0.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Re: Mouse click on wrong place with Edge

Post by TimoL » Tue Mar 17, 2020 6:56 am

It's old version Microsoft Edge 44.18362.449.0. Ranorex version is 9.2.0.

I noticed that the test works fine if I run the Edge iteration only. But if I run all iterations (there are browsers listed in the source file: IE, Firefox, Chrome, Edge), then in Edge test all the clicks are performed above the right elements.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6180
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Mouse click on wrong place with Edge

Post by odklizec » Tue Mar 17, 2020 8:50 am

Hi,

Unfortunately, it's hard to tell what's wrong without seeing it in action. Therefore, I would suggest to contact directly Ranorex support via support form.

Anyway, I think you should try new Edge and most recent 9.3, or at very lest, 9.2.1.

BTW, are you sure that all browsers are closed before starting Edge? Especially, please check if Chrome is completely terminated. By default, Chrome is still running in background, even if the main window is closed and so Ranorex plugin in Chrome is loaded. There is an option in Chrome settings to disable background run. I'm not telling that it's the cause of your issue, but it's worth a check. Just to eliminate this possibility. But I think it's something completely different ;)

Anyway, I thing that old Edge is now dead and because it never gained higher popularity, it's probably safe to simply ignore it? ;)
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Re: Mouse click on wrong place with Edge

Post by TimoL » Tue Mar 17, 2020 5:11 pm

I updated Edge to the latest version and Ranorex Studio to 9.3.0. Results:
-objects of Chrome are not recognized at all, even though it's on whitelist
-some mouse clicks on Edge happens too fast
-mouse clicks on EdgeChromium are performed above the right elements (but now closer than earlier)
-Edge and EdgeChromium opens in same browser

User avatar
Vega
Certified Professional
Certified Professional
Posts: 222
Joined: Mon Jul 11, 2016 2:15 pm

Re: Mouse click on wrong place with Edge

Post by Vega » Wed Mar 18, 2020 12:30 am

Hi,

Please share a Ranorex snapshot of any page in Chrome and or edge. Since you mention that objects are not recognized at all in Chrome, any page (such as Google or Ranorex.com) will work. This will let us see what Ranorex can see and often times can help spot the issue. You can learn to create a Ranorex snapshot here:

https://www.ranorex.com/help/latest/han ... -snapshot/

Also when you open Spy, make sure you are getting a DOM object. For web automation you should see two base elements in Spy, the DOM which is the website itself and the browser form object which is the browser controls. Its kind of like picture versus picture frame:
farumz.png
farumz.png (69.23 KiB) Viewed 292 times

In regard to your mouse clicks not occuring where you are expecting, please make sure your browser zoom level is 100%. You may also want to see if your DPI scaling for your OS is greater than 100%

hope this helps

TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Re: Mouse click on wrong place with Edge

Post by TimoL » Wed Mar 18, 2020 7:41 am

Chrome issue solved, but now Edge (or EdgeChromium) is not recognizing any objects. I'm a bit confused with whitelisting: there's whitelist1 when Ranorex Studio is started, but no project selected. I have understood this affects on standalone Spy. I had this empty earlier and everything worked fine. Then there's whitelist2 in the Ranorex Studio project. Here I have had listed all the browsers and everything worked fine, except clicks with Edge started to perform above the objects (this worked earlier).

Then I updated Ranorex Studio from 9.2 to 9.3 and Edge to latest version. After that Chrome objects were not recognized at all. I added chrome to whitelist1 (it was already in whitelist2) and after reboot, the Chrome objects were recognized again, but now Edge objects were not recognized anymore. I added Edge also to whitelist1 (it was already in whitelist2) and rebooted the machine, but it didn't help. Attached picture from Spy when Ranorex.com page is tracked.
Edge_ranorexpage_in_spy.PNG
Edge_ranorexpage_in_spy.PNG (38.51 KiB) Viewed 264 times
DPI scaling was 100% as well as browser zooming. One reason for click in wrong place could be RDP connection (I have noticed such sometimes), but as said, the objects are not recognized at all at the moment with Edge. IE, Firefox and Chrome works fine.

User avatar
Vega
Certified Professional
Certified Professional
Posts: 222
Joined: Mon Jul 11, 2016 2:15 pm

Re: Mouse click on wrong place with Edge

Post by Vega » Wed Mar 18, 2020 3:30 pm

Yes RDP is known to cause issues and or artificial latency especially if you have your mouse over the active RDP window. I would suggest another remote tool or simply do not keep the session open while running your test. There are several ways you can do this, but it depends on your environment. If you are not developing on the machine you are remoted into, then you can take advantage of something like Ranorex Agent. If you absolutely have to use RDP while running your test, then try starting your test and then minimizing the rdp window and focusing another window. I know it sounds weird but RDP can be really janky. If possible, I would just outright close the rdp window while you run your test but the problem is this will also close the user session and thus fail your test. You can use the keep alive script to keep the session open or better yet, utilize Ranorex Agent.

How are you opening Spy? If you open Spy from within Ranorex Studio with a project open, it will use your solution settings. If you are opening Spy from the start menu, shortcut, or without a project open, it will then use your global machine settings. I would recommend opening Spy from within Studio while your solution is open to keep your settings consistent. When a new solution is created, it uses your global settings as the template.

In the screenshot of Spy you provided, you are showing the Edge form object. This is basically the picture frame and will only really contain the browser controls. What you want to be looking at is the DOM object instead. Try removing all white listed processes, close all browsers except for one instance of Edge which has a page loaded (such as Ranorex.com or google). Then look in Spy and see if you are getting a DOM object; you can refer to my previous screenshot for reference.

Hope this helps

TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Re: Mouse click on wrong place with Edge

Post by TimoL » Thu Mar 19, 2020 3:42 pm

I use RDP only for tests development. The scheduled tests are run in local machine by Jenkins. Minimizing RDP window don’t work, because then it gets blank screen. But this Edge issue may be caused by RDP.

Thanks about clarifying the whitelists! I’m normally using standalone Spy. I didn’t make any changes, but now again Chrome elements are not recognized at all. I also cleared the global whitelist, opened the standalone Spy and Chrome and still the same: the whole page is recognized as one element only. No DOM object. And same if Spy opened from Studio.

After one reboot again, I got Ranorex technology limitation warning about Chrome add-on. I uninstalled it and installed (instrumented) successfully again and restarted the Chrome, but always the same warning or then the whole page is recognized as one element only.

Chrome is the browser where I have run tests successfully for a long time with several Ranorex updates and there hasn’t been this kind of problems before the update to Ranorex 9.3.

User avatar
Vega
Certified Professional
Certified Professional
Posts: 222
Joined: Mon Jul 11, 2016 2:15 pm

Re: Mouse click on wrong place with Edge

Post by Vega » Thu Mar 19, 2020 6:02 pm

How are you instrumenting the browser? Are you using the instrumentation wizard?

You can try different combinations of the below:
  • If you instrumented Chrome using the instrumentation wizard, try disabling the instrument property on your open browser action (Right click -> Properties)
  • Alternatively, remove the Chrome extension using the instrumentation wizard, but set the instrument property on your open browser action to true
Make sure you are fully closing all Chrome instances before installing or removing the extension. Chrome likes to say open in the background by default, so ensure it is not running as a background process.

If you are able to share a Ranorex snapshot of Chrome open to a website, it would be very helpful. However if this is a blocking issue for you, I would probably just open a support ticket:

https://www.ranorex.com/support-query/

hope this helps

TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Re: Mouse click on wrong place with Edge

Post by TimoL » Fri Mar 20, 2020 2:10 pm

Thanks Vega! Now the Chrome is working again. I had instrumented the plugin via Instrumentation wizard. Now I set the Instrument property of Open browser as false and it works.

Now there's the same problem with Edge: Ranorex Studio or Spy recognizes it just as one element: Container 'Intermediate D3D Window'. I instrumented the Edge via Instrumentation wizard, but it didn't help. Earlier without instrumentation by Instrumentation wizard, but with Instrument property as true, the elements were recognized, but a bit above the right places. Edge is not that important for me, but I will contact Ranorex support if I don't find a solution for that.

TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Re: Mouse click on wrong place with Edge

Post by TimoL » Tue Mar 24, 2020 6:19 am

It seems that Chrome and Edge add-ons can't be instrumented at same time: I succeeded to instrument Edge add-on after several try (Instrumentation wizard's success notification does not guarantee that). Then Edge objects were recognized, but Chrome objects not anymore. I instrumented (again) Chrome add-on and rebooted the machine: Chrome objects were recognized, but Edge objects not anymore.

Is it really so that you can't have both Chrome and Edge instrumented simultaneously?

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6180
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Mouse click on wrong place with Edge

Post by odklizec » Tue Mar 24, 2020 8:19 am

Hi,

Are you using latest 9.3.1? There was a Chrome plugin related bug in 9.3.0, which was fixed in 9.3.1.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

TimoL
Posts: 33
Joined: Thu Sep 13, 2018 3:08 pm

Re: Mouse click on wrong place with Edge

Post by TimoL » Tue Mar 24, 2020 12:47 pm

I updated to 9.3.1 and noticed that Chrome objects are found only if there's no Edge background processes running and Edge objects are recognized only if there's no Chrome processes running. Otherwise the whole window is treated as one element only. Seems that the Ranorex plugin is same for Chrome and Edge.

After denying the background processes (Settings -> System -> Continue running background apps when Microsoft Edge(/Chrome) is closed), everything works finally fine. Thanks odklizec!

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6180
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Mouse click on wrong place with Edge

Post by odklizec » Tue Mar 24, 2020 12:52 pm

Great to hear that! Continuous running of Chrome apps in background is a source of many issues. It should be by default off ;)
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration