Test Cases Failed on VM

Ask general questions here.
c676228
Posts: 176
Joined: Mon Apr 06, 2015 5:40 am

Test Cases Failed on VM

Post by c676228 » Thu Sep 10, 2015 1:14 am

Hi,

I just moved my automation project to the VM. Things are getting uneasy.
Automation test cases can be successfully completed on my physical dev machine are failed easily due to timing issue. Even worse, the behavior is pretty random and inconsistent. Don't know how to deal with it. First of all, I adjust time factor from 1 to 1.1 in Global Settings.
However, it doesn't help much. It is still failing. Meantime, I already slowed down the automation in my project since it is a web automation project and response time could be longer than usual from time to time.
So it is even more painful on VM.

Any better suggestions? Will browser plugins cause slowness on VM even more?

Thanks,
Betty

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3930
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Test Cases Failed on VM

Post by odklizec » Thu Sep 10, 2015 7:40 am

Hi Betty,

Automatic tests of HTML is always tricky. Artificial delays are often not good enough and only make the tests unnecessary slow. What you should use (instead of slowing your test down) is a combination of WaitForExists/NotExists (for individual repo elements), WaitForDocumentLoaded (for DOM elements) and eventually @state='complete' applied in DOM xpath. It's not necessary to use WaitForExists for every element on page. You should rather use it for most problematic elements, which you know may fail because of longer element loading or page processing time. There is no general rule how to apply these methods in your tests. At first, apply them everywhere you are now using Delay actions ;) Then add some more of them at places, where your scripts usually fail. Hope this helps?
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

c676228
Posts: 176
Joined: Mon Apr 06, 2015 5:40 am

Re: Test Cases Failed on VM

Post by c676228 » Thu Sep 10, 2015 5:54 pm

Pavel,

Thanks for your input.
That definitely makes sense to me. I do use waitforExist method a lot. The reason I don't use it all the time is sometimes the WaitForExist method doesn't work all the time so I just use hard delay. I will go through my code to use WaitForExist for long delay at least for now.

However, I am wondering if you experienced actual object recognition issue on VM? I noticed sometimes it is not timing issue, when the object is available, the object cannot be located or identified timely.
This can be proved by my code since it is a loop.
1) The first two iteration runs ok, on the third iteration, it suddenly says that the element is not found.
2) When I use Ranorex Studio on VM to update my code, the screen just cannot display intellisense info properly.
Basically I cannot see the parameters info etc.
I mean the VM display issue cause the object identification failed.
Any suggestions?

Regards,
Betty

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3930
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Test Cases Failed on VM

Post by odklizec » Fri Sep 11, 2015 8:48 am

Hi Betty,
[quote="c676228"However, I am wondering if you experienced actual object recognition issue on VM? I noticed sometimes it is not timing issue, when the object is available, the object cannot be located or identified timely.[/quote]
I've experienced problem like this in the past with Java-based application.

In some cases, WaitForNotExists failed for me, because Ranorex somehow detected the element (status bar text) no longer exists, while it was still available on the screen. I found that this was Java-based GUI related issue.

I solved it by writing my own WaitForNotExist method, which detected the existence of element in a loop. If method detected the element no longer exists, it went to a second "timed" loop, where I tested the existence/not existence of the element every 100ms for a defined period of time (e.g. 2000 msecs). If the element reappeared during the defined period, it jumped out of the timed loop and continued with the main WaitForNotExist loop. If the element has not been found during defined timeout, I considered the element definitely not existing ;)

I think it should be possible to create similar method for WaitForExists. Maybe it would solve your problem?
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

c676228
Posts: 176
Joined: Mon Apr 06, 2015 5:40 am

Re: Test Cases Failed on VM

Post by c676228 » Tue Sep 15, 2015 9:09 pm

Pavel,

Thanks for the reply. So far I haven't written any methods to replace Ranorex methods. I still need time to explore. However I did report some bugs to Ranorex.

Here are what have been done to the VM and it does help to improve the performance related and object identification issue.

1) Increase from one to four processors
2) adjust rdp display from high quality 32bit to true color 24bit( and full screen mode)
3) Apply waitforexist or waitfornonexist whenever it is proper

It did get much better. But I do have some other weird issues I have never encountered when I am using real machine. I will post them separately.

Regards,
Betty

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Test Cases Failed on VM

Post by krstcs » Tue Sep 15, 2015 9:43 pm

One thing that I would suggest is using a VNC clone of some kind instead of RDP when logging into the systems. RDP is problematic for many reasons and VNC will allow you to disconnect while leaving the VM with an active desktop.

I use TightVNC, but there are many others out there.

You can set the VNC installation to always have a desktop available, as if it were on 'real' hardware.

I was resistant at first, but after using it for a while, I don't see how I could do what I do without it. RDP is just lacking severely in comparison.
Shortcuts usually aren't...