Tests run from Azure DevOps not starting apps on VM

Ask general questions here.
Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Wed Feb 05, 2020 11:54 am

I have set up a Pipeline and agents in Azure DevOps, to run Ranorex (9.2.1) tests on a Windows 10 VM. The Ranorex tests can be run from the Azure Pipeline and the test report can then be viewed on both the VM and on Azure. The pipeline is also successfully checking out the Ranorex solution from Azure (Git repo) to the agent on the VM.

My issue is that when the tests are run from the Azure Pipeline, they do not open or interact with apps on the VM. E.g. I have a test to open Windows File Explorer. When I run that test from Azure, it does NOT open File Explorer on the VM, although the Ranorex test report shows that the test has run.

When the same test is run from the Ranorex solution on the agent, on the VM, (which has been successfully checked out via the Azure Pipeline test run), the test does open File Explorer.

Would you please have suggestions as to what could be causing this issue or how to resolve it?

Thanks

McTurtle
Posts: 294
Joined: Thu Feb 23, 2017 10:37 am
Location: Benedikt, Slovenia

Re: Tests run from Azure DevOps not starting apps on VM

Post by McTurtle » Wed Feb 05, 2020 12:56 pm

Hi Fergal,

Two things you can try:
1. Enable the UiaLauncher in "Program.cs" by un-commenting these lines:
UiaLauncher.png
UiaLauncher.png (15.69 KiB) Viewed 246 times
2. Make sure there is an active session on the VM when running the test via Azure. Does the same happen if you are "watching" the RDP window when the test is started by Azure?

Regards,
McTurtle

Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Wed Feb 05, 2020 2:32 pm

Thanks very much McTurtle.

When those two lines are un-commented, the test solution still runs correctly in Ranorex, on the machine where the tests are created. However, it fails on the Azure Pipeline Build, at the command line task, with the error message
cmd [dot e x e] exited with code '111'
This forum / Cloudflare is not liking the file extension in the quote above

After the Azure build I can open the test solution from the agent on the VM and see that the lines are un-commented. When that solution, from the agent, is run directly on the VM via Ranorex, it runs exactly as it should.

When those two lines are commented out again, the solution runs from Azure, but still doesn't open the apps on the VM.
McTurtle wrote:
Wed Feb 05, 2020 12:56 pm
Does the same happen if you are "watching" the RDP window when the test is started by Azure?
Yes it does, when I watch the VM during the test run from the Azure Pipeline, nothing happens at all.

Thanks again.

McTurtle
Posts: 294
Joined: Thu Feb 23, 2017 10:37 am
Location: Benedikt, Slovenia

Re: Tests run from Azure DevOps not starting apps on VM

Post by McTurtle » Wed Feb 05, 2020 2:38 pm

Hey!

Take a look at this: Error code 111

Can you try to shorten the path to the Ranorex exe or the path to the AUT? Something is a too long string, it seems.

Regards,
McTurtle

Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Wed Feb 05, 2020 5:17 pm

Thanks again McTurtle.

The log is also saying:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Ranorex.Core.RanorexCoreException: Failed to launch the executable with UI access rights. ---> System.ComponentModel.Win32Exception: This operation requires an interactive window station
Perhaps something needs to be changed on the VM?
Can you try to shorten the path to the Ranorex exe or the path to the AUT?
The path to the Ranorex exe doesn't seem very long, also I'm not sure that is something I can alter.

The build also fails if I disable the test that starts the AUT, so not sure if that is the issue. I will give your question more thought and see if I can figure something out.

McTurtle
Posts: 294
Joined: Thu Feb 23, 2017 10:37 am
Location: Benedikt, Slovenia

Re: Tests run from Azure DevOps not starting apps on VM

Post by McTurtle » Thu Feb 06, 2020 6:58 am

Hi Fergal,

Ok, obviously you should undo that change to the "program.cs" file from my 1st post :) The error is saying it can't do that.
Which error do you see int he same log when that change is undone and you try to run via Azure again?

Also, you are sure you did set the agent to run as a process and not as a service?

Regards,
McTurtle

Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Thu Feb 06, 2020 1:27 pm

Thanks McTurtle.
McTurtle wrote:
Thu Feb 06, 2020 6:58 am
Ok, obviously you should undo that change to the "program.cs" file from my 1st post :) The error is saying it can't do that.
Which error do you see int he same log when that change is undone and you try to run via Azure again?
There is no error at that point. The solution is checked out to the agent on the VM. The tests are run, but they don't open or interact with any apps on the VM. The Azure tasks all run successfully. The Ranorex tests fail because the app is not open. The Ranorex report shows nothing but a black square as the screenshot, at the point it fails because the app is not open.

If I open the Ranorex solution, which Azure checked out to the agent on the VM, then run that solution directly on Ranorex, it runs exactly as it should.

McTurtle wrote:
Thu Feb 06, 2020 6:58 am
Also, you are sure you did set the agent to run as a process and not as a service?
Thanks I've double checked and the agent is not running as a service.

McTurtle
Posts: 294
Joined: Thu Feb 23, 2017 10:37 am
Location: Benedikt, Slovenia

Re: Tests run from Azure DevOps not starting apps on VM

Post by McTurtle » Thu Feb 06, 2020 1:39 pm

Hi Fergal,

When you say "The tests are run", do you mean that a console window opens with "Console logger starting" and "Run application" and all that Ranorex related messages? Also, the progress dialog appears? This is very important. Because, if this does not appear, then the Agent is not starting the test under the same user session as the one that you are currently logged in on the computer.

If you say there is a "black square" on the screenshot, do you mean that the screenshot is a black square? Can you post an image of it here on the forum?

Regards,
McTurtle

Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Thu Feb 06, 2020 2:06 pm

McTurtle wrote:
Thu Feb 06, 2020 1:39 pm
...When you say "The tests are run", do you mean that a console window opens with "Console logger starting" and "Run application" and all that Ranorex related messages? Also, the progress dialog appears?..
No.
When the tests are run from Azure, the Ranorex solution is checked out to the VM and a Ranorex report is created with the test results. E.g.
1. A module that contains nothing but a delay (added just to test the process) is reported as passing
2. A module that opens an app, but does not contain validation, is reported as passing
3. A module that clicks on a button on the app fails, because the app is not opened.

When I view the VM, while the tests are running, nothing at all happens.

The progress dialog is disabled in the test suite properties.
a console window opens with "Console logger starting"
I can see that in the Azure log, but nothing appears on the VM.

Image
Yes the screenshot is a black square.

One thing I have noticed is that the name of the Computer/Endpoint on the Ranorex test report is the correct name of the VM where the test should run.

Thanks again McTurtle, I really do appreciate you looking at it in such detail.

Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Thu Feb 06, 2020 4:58 pm

Also, when I visit the VM, with Ranorex closed on the machine, then double click on the solution exe file in the project's bin > Release folder (from the Agent where Azure has checked out the solution), the test runs correctly.

This suggests to me, that Azure is checking out the solution as it should, but not starting the run of the exe file correctly.

Thanks

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 5935
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Tests run from Azure DevOps not starting apps on VM

Post by odklizec » Thu Feb 06, 2020 5:06 pm

Hi,

Are you sure the user is logged in while the test is running on VM? You see, the user must be logged in and desktop stay active (not locked) otherwise, Ranorex is not able to interact with UI elements! And this most probably also why there are black screenshots in your report. When you log to the VM (I guess via Remote Desktop?) and start your test manually, it will work fine. But once you close Remote Desktop window, your test will definitely fail as well, because closing or minimizing RD session locks the desktop! I’m personally using tool called Logon Expert, which automatically logs the user (once the VM is started) and keeps the session active, no matter what.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Thu Feb 06, 2020 5:20 pm

Thanks very much odklizec. I have logged into the VM, opened a browser on the VM and started the Azure Pipeline build from that browser, so definitely logged in, but still nothing happens on the VM and still black screenshots.

Does any particular user need to be logged into the VM? I mean, does the VM user account have to be connected to Azure DevOps in some way?

Thanks again.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 5935
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Tests run from Azure DevOps not starting apps on VM

Post by odklizec » Thu Feb 06, 2020 5:40 pm

Hi,

Is the build machine the same as the machine where you actually run your test? The active session and logged user must be on the test machine. And as mentioned before the agent controlling the test VM, must not be started as service!
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

Fergal
Certified Professional
Certified Professional
Posts: 333
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Tests run from Azure DevOps not starting apps on VM

Post by Fergal » Thu Feb 06, 2020 5:52 pm

Thanks odklizec.
odklizec wrote:
Thu Feb 06, 2020 5:40 pm
Is the build machine the same as the machine where you actually run your test?
No, they are different machines.
odklizec wrote:
Thu Feb 06, 2020 5:40 pm
The active session and logged user must be on the test machine.
This is similar to what McTurtle said.
McTurtle wrote:
Thu Feb 06, 2020 1:39 pm
...the Agent is not starting the test under the same user session as the one that you are currently logged in on the computer...
Unfortunately I don't fully understand that. Where can I learn more about that or what can I do to resolve that issue?
odklizec wrote:
Thu Feb 06, 2020 5:40 pm
...the agent controlling the test VM, must not be started as service!
The agent is not being started as a service.

Thanks again.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 5935
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Tests run from Azure DevOps not starting apps on VM

Post by odklizec » Thu Feb 06, 2020 5:58 pm

Try this... log on the test VM via Remote Desktop and keep the RD window open. Then start the test via azure pipeline, as you normally do. Just make sure the test is started with the same user session/credentials.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration