Running from IDE tests pass, from agent tests fail

Ask general questions here.
martijndevrieze
Posts: 8
Joined: Mon Sep 23, 2019 11:35 am

Running from IDE tests pass, from agent tests fail

Post by martijndevrieze » Wed Oct 21, 2020 7:11 am

I am running Ranorex as a testautomation tool against Microsoft Dynamic AX.
When I run my tests from within the IDE the tests invariably pass. However the moment I run the tests as a stand alone executable or from a remote agent the tests fail.
When run stand alone the tests somehow do not recognize objects, or even entire screens.
The odd thing is that this even happens within 1 Windows 10 machine:

1. run tests from within Ranorex IDE
2. rebuild the tests to an exe
3. trigger the executable directly

In option 1 the tests pass
In option 2 the tests fail

If anyone has any clue as to what may have changed or happened, please let me know.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6560
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Running from IDE tests pass, from agent tests fail

Post by odklizec » Sun Oct 25, 2020 9:05 am

Hi,

Does both events happen on the same machine?

What Ranorex version do you use? If something older than most recent 9.3.3, then update it.

Could you please add Create Snapshot action, just before the typically failing command? And in this action specify main application form of the app. Then run the application both ways and share both resulting reports with us. In case they are bigger than forum upload likes, use a 3rd party file sharing service and share a link.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

martijndevrieze
Posts: 8
Joined: Mon Sep 23, 2019 11:35 am

Re: Running from IDE tests pass, from agent tests fail

Post by martijndevrieze » Wed Nov 04, 2020 10:17 am

I am indeed right now still on version 9.3.2 so will start with upgrading to the latest version.
After that I will drop an update here again.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6560
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Running from IDE tests pass, from agent tests fail

Post by odklizec » Wed Nov 04, 2020 10:29 am

Hi,

Although it's always good to keep Ranorex updated, I don't think it will help in this particular case. I think you need to compare snapshots created during both runs (IDE and agent). I guess that there is a difference in UI, causing your problem. But it's impossible to tell what's wrong without seeing, at very least, snapshots ;)
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

martijndevrieze
Posts: 8
Joined: Mon Sep 23, 2019 11:35 am

Re: Running from IDE tests pass, from agent tests fail

Post by martijndevrieze » Wed Nov 11, 2020 10:39 am

Ok...
I am probably doing something wrong, but when running the test from within the IDE I get a snapshot, when running the exe I do not get a snapshot

I added it as follows:
Image

During the exe run the test does not see the object. When I break the test and open the Spy it does see the object without fail.
Attachments
snapshots_OpenEenK_95535163.zip
(1.08 MiB) Downloaded 11 times

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6560
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Running from IDE tests pass, from agent tests fail

Post by odklizec » Wed Nov 11, 2020 11:06 am

Hi,

At first, there is an instrumentation error displayed in snapshot, which most probably affects Ranorex ability to consistently recognize UI elements in your app under test. So you should follow the description of this error in User Guide and do whatewer it says to clear this error.
2020-11-11_10-59-32.png
2020-11-11_10-59-32.png (152.44 KiB) Viewed 257 times
As for the problem of getting snapshot when running exe, I would suggest to get snapshot of entire form, instead of this particular Element12583, which I guess is simply not found, because of a difference in UI, possibly caused by the above warning? So get the snapshot or this form instead:
/form[@title>'‪Klanten‬']
But as mentioned, you should first eliminate the instrumentation error.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

martijndevrieze
Posts: 8
Joined: Mon Sep 23, 2019 11:35 am

Re: Running from IDE tests pass, from agent tests fail

Post by martijndevrieze » Wed Nov 11, 2020 12:12 pm

That error is all nice, but I cannot do anything about what it complains about.
Here are the "solutions" proposed by the troubleshooting guide:
Mixed-mode executable/obfuscated controls

The controls that can’t be identified are implemented in a mixed-mode executable (not DLL). These are often created by obfuscating applications or assembly merging utilities. The .NET Framework doesn’t support loading such mixed-mode executables in other processes, which is why Ranorex Studio and its tools cannot recognize such controls.

Solution: As a workaround, you can try to automate the non-obfuscated version of your application.
The application is not obfuscated, since from within IDE with the Spy I CAN see the objects. I am not starting the application any differently when running tests from within the exe compared to when run via IDE.
So If you have any ideas or suggestions what i can try to resolve this error please feel free.

Again, this is Microsoft Dynamics AX 2012 I am running against. Nothing changed in that. The only difference between before and now is the CPU of the Virtual machines I am using. For the rest it is the same user, same rights, same Windows patch levels, same client which is started in the same way as before.

In the meantime I will try to get the snapshot of the overall object rather than the detail I am looking for. The validation of the popup including its title-bar does work fine btw.
I still do not see how the snapshot DOES generate when run from IDE and does not generate when run as clean executable. It seems, in the messaging/logging as if the test does not even reach that point. All I see is a message "Waiting for 0ms", then 30 seconds nothing and then the test fails.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6560
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Running from IDE tests pass, from agent tests fail

Post by odklizec » Wed Nov 11, 2020 12:28 pm

Hi,
I understand your frustration from this issue, but it's pretty hard to suggest something sensible without seeing the app under test. I think you will have to contact Ranorex support and request a remote session/teamviewer with them, so they can see and analyze the issue directly with your app?
martijndevrieze wrote:
Wed Nov 11, 2020 12:12 pm
I still do not see how the snapshot DOES generate when run from IDE and does not generate when run as clean executable. It seems, in the messaging/logging as if the test does not even reach that point. All I see is a message "Waiting for 0ms", then 30 seconds nothing and then the test fails.
The crucial question is, with what error the test fails? Could you please provide entire report, where we can see exact error? My guess is, that Ranorex fails to find the element and therefore, it cannot create the snapshot. And if it cannot find the element, it's because the xpath used in repo is not the same as the xpath of "live" element.

The snapshot you posted was obtained from test started from IDE, right? Could you please create also snapshot via standalone Spy? Ideally, create two snapshots with standalone Spy. One with Spy started as administrator, the other one without administrator privileges. BTW, are you running Ranorex Studio as administrator?
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

martijndevrieze
Posts: 8
Joined: Mon Sep 23, 2019 11:35 am

Re: Running from IDE tests pass, from agent tests fail

Post by martijndevrieze » Wed Nov 11, 2020 12:30 pm

If there were an error I would have given it :)
I do not see any error or any mention with regards to the snapshot. As if it isn't there in the exe version.
Going to dig around and see if I can make the snapshot of the entire screen first and see if I then get more output and/or an error.

martijndevrieze
Posts: 8
Joined: Mon Sep 23, 2019 11:35 am

Re: Running from IDE tests pass, from agent tests fail

Post by martijndevrieze » Wed Nov 11, 2020 1:55 pm

Finally managed to get a snapshot whilst running from the exe and it turns out the panels and screens when running from the executable have an actual name rather than numbers.
So next up: figuring out whether I can find these names when running in IDE as well and more importantly, what changed that the objects have different values when running in different modes.

The moment I manage to pull the snapshots from the machine I will add them here as well.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 6560
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Running from IDE tests pass, from agent tests fail

Post by odklizec » Wed Nov 11, 2020 2:19 pm

Hi,

Do you have settings stored in solution?...
2020-11-11_14-17-19.png
2020-11-11_14-17-19.png (38.45 KiB) Viewed 245 times

If not, this could eventually explain different behavior if started from IDE and standalone exe.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

martijndevrieze
Posts: 8
Joined: Mon Sep 23, 2019 11:35 am

Re: Running from IDE tests pass, from agent tests fail

Post by martijndevrieze » Thu Nov 12, 2020 2:04 pm

I indeed have that setting as well, so that, theoretically cannot be the problem.