Limiting concurrent Jenkins jobs base on # of Rx Licenses

Experiences, small talk, and other automation gossip.
tvu
Posts: 195
Joined: Tue Apr 07, 2015 10:47 pm

Limiting concurrent Jenkins jobs base on # of Rx Licenses

Post by tvu » Fri May 22, 2015 8:23 pm

This is probably a better topic on the Jenkins forum, but I am hoping someone has already dealt with it here.

I use Jenkins to automate my build process and kick off my Ranorex tests. I am looking for a way to limit the number of concurrent Jenkins jobs across the entire configuration (master + slaves) based on the number of Ranorex licenses. I am not talking about parallel runs of the same job, but parallel runs of independent jobs that will use the Ranorex license.

I am already using the Locks and Latches plugin to limit jobs based on a single shared resource, i.e. one Simulator / Emulator. But, it doesn't work in this case because there are multiple licenses to share.

Example setup:
- Jenkins: 1 x master, 2 x slaves (each node have 4 executors)
- Ranorex Licenses: 2 x Premium Floating, 4 x Runtime

I can have multiple Ranorex jobs running at the same time:
- Two Devs have Ranorex Studio open and using the two Premium Floating licenses
- Master Node - running Job1
- Slave1 Node - running Job2, Job3
- Slave2 Node - running Job4
- Waiting for free licenses - Job5, Job6, Job7

When Job2 is complete, Job5 will start on one of the nodes.

Can the Premium Floating licenses be used to run the Ranorex tests? I am new to Ranorex so I am not sure about this. If that's possible, it would awesome to setup Jenkins to limit the concurrent jobs during the day to the number of Runtime licenses. Then at night, increase that number to include the Premium Floating licenses.

Thanks for the help,
Tung

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 7470
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Limiting concurrent Jenkins jobs base on # of Rx Licenses

Post by odklizec » Tue May 26, 2015 7:19 am

Hi,

Yes, you can use Premium licenses to run your tests. In Ranorex license manager select "Auto" type instead of choosing specific license type (Runtime, Professional, Premium). This instructs Ranorex to use license for whatever requested task (runtime or studio). The downside of this solution is that runtime job could easily "eat" studio license ;) But I believe that Ranorex first tries to use runtime licenses for runtime jobs and only if there is no more runtime license available, pro/premium license is used?

Unfortunately, I don't think you can instruct Jenkins the way you want (to use runtime licenses for day tests and runtime+premium for night tests). The only thing you can probably do is to limit the number of concurrent executors? But I'm not sure if this setting could be parametrized in job settings? So maybe you will have to use two jobs (one for day, one for night tests), each with different number of concurrent executors?

Additionally, I would recommend you to use WaitForValidLicense at start of your Ranorex scripts. This could prevent the test failure in case of not available license. I'm using below method at start of program.cs
public static int Main(string[] args)
        {
            // check if Ranorex license is available (if not, wait for 15mins/check every 5secs)       
            WaitForAvailableLicense(900000,5000); 
            ...
            return error;
        }
 
	/// <summary>
	/// This method checks and waits for available RX license
	/// </summary>
	/// <param name="WaitForLicenseTimeout">wait time for available license (before giving up)</param>
	/// <param name="WaitForLicenseInterval">license check interval</param>
	/// <returns></returns>
	public static int WaitForAvailableLicense(int WaitForLicenseTimeout, int WaitForLicenseInterval)
	{
		System.Console.WriteLine("Waiting for RX license...");
		bool RXLicenseAvailable = Ranorex.Core.ElementEngine.WaitForValidLicense(new Ranorex.Duration( WaitForLicenseTimeout), new Ranorex.Duration(WaitForLicenseInterval));
		if (RXLicenseAvailable)
		{
			System.Console.WriteLine("RX license OK!");
			return 1;                    
		}
		else
		{
			System.Console.WriteLine("No RX license available! (waited " + TimeSpan.FromMilliseconds(WaitForLicenseTimeout).TotalMinutes.ToString() + " mins)");
			return 0;
		}
	}
Hope this helps a bit? ;)
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

krstcs
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Limiting concurrent Jenkins jobs base on # of Rx Licenses

Post by krstcs » Tue May 26, 2015 1:33 pm

First and foremost: You should only have 1 executor running on any of your slaves if they are going to be running Ranorex tests. You should NOT run multiple Ranorex tests on the same system at the same time as this can have unforeseen side-effects. Ranorex is designed for Functional UI automation, using the mouse and keyboard, and since no system can handle more than one mouse/keyboard driver at a time, there should only be one Ranorex test running at a time. The easiest and best way to avoid this is to not have more than 1 executor per test system.

Second, if a single system IS running multiple instances of Ranorex, they all use the same license, so you only need one license per active SYSTEM, not per active Ranorex application.

Third, Pavel is correct, Jenkins has no way of knowing how many license you have unless you tell it. Again, the best way to manage this is by limiting executors in Jenkins to one per system.
Shortcuts usually aren't...

tvu
Posts: 195
Joined: Tue Apr 07, 2015 10:47 pm

Re: Limiting concurrent Jenkins jobs base on # of Rx Licenses

Post by tvu » Tue May 26, 2015 4:56 pm

Thanks for the advice krstcs and odklizec!

The little tidbit regarding the active system vs active application is extremely useful.

We are using Ranorex mostly for mobile testing. When we are running tests that does not require any interaction with desktop application should we still limit the number of Ranorex execution to just one? Shouldn't it be possible to run parallel tests on different mobile devices?

For example, if we have a test suite that does UI validation on our mobile application. All it will do is go through all the menus to make sure everything is there. This test will not have any mouse/keyboard interaction. Is it possible to run this test on multiple devices at the same time from the same Jenkins Slave?

Thanks,
Tung

krstcs
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Limiting concurrent Jenkins jobs base on # of Rx Licenses

Post by krstcs » Tue May 26, 2015 5:39 pm

The official stance from Ranorex has always been that you should not run multiple Ranorex tests on the same system at the same time.

Since Ranorex is still running the test on the Windows system (it's just passing the actions through to the device's Ranorex service), I would be concerned about possible side-effects if you are running more than one at the same time.

It might be possible, but I wouldn't recommend it.
Shortcuts usually aren't...