Low performance when running tests on VM

Ask general questions here.
elena.ungureanu
Posts: 19
Joined: Mon Jan 05, 2015 5:46 pm

Low performance when running tests on VM

Post by elena.ungureanu » Mon Jul 20, 2015 11:18 pm

Hi,

I'm working on a testing framework built on top of Ranorex API. I've noticed that the execution time is at least 2 times higher on a dedicated VM compared to local machine.

I was wondering if you have any performance tips for running tests on a VM.

Thank you,
Elena

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Low performance when running tests on VM

Post by krstcs » Tue Jul 21, 2015 2:30 pm

I run all of my test on VMs and I don't see any difference in run times.

It could be any number of things, including how the VMs are set up, what the host is, how much RAM there is, etc. Without more information about your environment it will be hard to help.

Make sure the VMs meet the minimum specifications for Ranorex and have plenty of RAM and disk space allocated.

Additionally, make sure the host system is not over-taxed by having the VM running as this can slow down the host and the VM.


Also, please always include the following when raising issues:

Ranorex version
Windows version (where test is running or being developed)
Technology of SUT (HTML, Java, Flash/Flex, WPF, etc.)
RanoreXPath and snapshot of the element in question
Shortcuts usually aren't...

elena.ungureanu
Posts: 19
Joined: Mon Jan 05, 2015 5:46 pm

Re: Low performance when running tests on VM

Post by elena.ungureanu » Fri Jul 31, 2015 5:38 pm

Hi krstcs,

Sorry for the lack of details about environments and SUT

Characteristics of the HyperV hosting the VMs:
4 CPUs, with 12 Cores each, totalling 48 processor cores @1.89 GHz each
128GM of RAM
C: 558GB
E: 1.63TB

Characteristics of the VM were tests are running:
Win 7 Ultimate
4 cores, 4GB, processor 1.90GHz

.NET Framework 4.5.1
Ranorex 5.2.1

The application under test has a web component (HTML/javascript) and a WPF component. Test for the web part are using Chrome.

Can you please give me more details about what
"the host system is not over-taxed by having the VM running"
means? Is not clear for me.

Thank you,
Elena

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Low performance when running tests on VM

Post by krstcs » Fri Jul 31, 2015 5:43 pm

I just meant that you needed to make sure the host had enough CPU power and memory to adequately run the guest VMs. It looks like yours does. :D

Have you tried updating Ranorex to a supported version? 5.2.1 is very old and is no longer supported. There have been numerous updates and bug fixes, any number of which could be affecting your tests.

Update to 5.4.0 and see if that helps.
Shortcuts usually aren't...

mrollins
Posts: 10
Joined: Wed Aug 28, 2013 6:35 pm

Re: Low performance when running tests on VM

Post by mrollins » Thu Aug 06, 2015 12:47 pm

WPF performance is abysmal in some VMs, particularly Hyper-V. The main reason is that the many VMs lack the hardware acceleration that WPF uses to make fonts, animations and scaling graphics work smoothly.

If you can have your developers add a switch to enable software rendering only mode in your WPF app it can speed things up slightly. Another thing I had my developers do was add a config switch to disable all animations and fade in/outs. That vastly sped up automation for our WPF apps.

As a bonus we've had customers turn on the same options on low spec computers to avoid having to upgrade employees' computers. It's win-win.

--
Michael