Page 1 of 1

Inconsistent Playback

Posted: Fri Oct 11, 2013 1:12 pm
by Mozzytm
Hi,

We invested in licences for Rx however its giving us some substantial headaches and i wonder if you can help. Our issues are mainly consistant playback as we are in a scenario where the software cant be trusted to run unmonitored.

We are using the software to automate a desktop client and a series of web apps. What is happening is we will record, tweak and playback a test, confirm it as fit for purpose and file it. When we come to run it you have about a 1/10 chance that it will actually make it to the end without intervention. Scenarios where it randomly goes wrong are:

entering text, sometimes it seems to miss off the first couple of characters of the entered string
(in this case ive made it SLIGHTLY more reliable by putting {DELETE}{DELETE} before the keypress event

finding on screen elements (web), these elements are not dynamically named so the repository entry should always stand however on some run throughs it will just bail as it cant find button X or div Y

(very rarely) Starting Firefox - in some cases it does not even start the browser

track - We were having problems clicking a button on a popup window in web, we could see the button and track would seem to highlight it and give a repo entry that all checked out green yet it just WOULD-NOT click the button. we had to fudge a way around it by using tabs and space/enter

Validation - in an effort to minimise delays we use validation with a timeout however sometimes the item in question will be present yet the validation will just not see it so the script fails, we've had to go back to using clunky delays


There are other scenarios but these are the main ones and help to increase stability would be appreciated

Just to be clear, these are consecutive run-throughs on the same PC with no other apps open in the space between tests. There just doesn't seem to be any rhyme or reason to it.

Thanks

-M

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 1:42 pm
by krstcs
First, can you post a test project or code that demonstrates the specific problems?

Second, is this your first attempt at automation with Ranorex? If so, I highly recommend reading the user manual (http://www.ranorex.com/support/user-guide-20.html) if you have not.

Several of the things you list as problems sound more like misunderstandings about how Ranorex works and how it interacts with objects in your system-under-test.

Timing issues in the recordings are often due to not having delays in the test. As fast as most computers and software are, they will still have delays in presenting/loading information that you will have to account for in your test script. And Ranorex will happily move on to the next step in the test regardless of what the SUT is doing, unless you tell it to wait for something specific.

Remember, computers have to be told EVERYTHING that you want them to do, including waits, exactly how you want them to do it. If you, as a human tester, have to wait on a page reload, then Ranorex needs to be told to wait on a page reload, for example.

Also, Ranorex uses MSAA or other accessibility attributes to identify elements, so if that is not implemented correctly, you will have issues.

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 1:43 pm
by Swisside
Hello

I don't have much experience with Ranorex yet but already experienced some inconsistency too. Here is how I (partially) solved some issues :

1) Redefine the RxPaths (I have no idea why but a more precise path worked better for me) for some repo items.

2) You already did it but I'm currently combining repo items and simple keyboard actions to reduce the delays and the inconsistency (some UI elements just take forever to be recognized)

3) Using click based on position (if you run the test on the same machine with the same resolution you can sometimes give coordinate and save some time, but it's risky at the same time)

4) Regarding validation I ended up coding all of it since the validation feature wasn't working well. However I don't know what you need to validate. (My cases are more data driven)


I'm only automating a desktop client so my tricks might not work for the web apps.


It's not much but I hope it will help you.


Boris

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 1:58 pm
by krstcs
I would highly recommend NOT using location-based identification unless it is absolutely necessary. This makes it much harder to maintain the tests in case the developers change something (or actually WHEN they do).

Again, I think many of the issues are timing issues and perhaps revolve around an incomplete understanding of how Ranorex works. Especially for web-based SUTs, there are issues if you don't wait for the pages to load fully.

Ranorex isn't perfect, but it is very, very good, and it is very a complex tool. It can take a bit to really get the hang of how to make it work for a given technology under test.

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 2:07 pm
by Swisside
I know still when everything else fails it's still better to have a test working than nothing at least temporarily.

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 2:10 pm
by krstcs
That is true! :D

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 2:20 pm
by Mozzytm
i genuinely dont think this is an ignorance issue, i am ensuring my web pages are fully loaded by a combination of validation and delays. id prefer to have used the readystate property but apparently (after a good bit of research) this is a non starter.

Ive been using Rx on and off since Feb so i'm aware of some of its quirks and have been pretty much machine gunning delays into scripts that are over compensating just to try and ENSURE it hits its mark but no go. (for example a 5 min delay where a 1min would be sufficient)

I disaggree that its best to have a test partially working than nothing, as if its partially working i still have to sit at the pc and watch it and to that end i may as well be doing the clicks myself.

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 2:53 pm
by krstcs
OK, so back to my original question. Can you post the code so we can get eyes on it? And, can we get a snapshot of the application so we can understand what the structure looks like?

Right now I feel like a mechanic trying to diagnose car problems over the phone based on the sounds the driver is making... :D


And, I wasn't meaning to insult you, hope it didn't come across that way. If it did, my apologies. I just took from your original post's lack of specificity that you weren't quite sure how it was working, so I went from there.


1. Did you try using a mouse Click() on the text field? For some reason this helps with some of the text entry issues.

2. What do you mean by "bail"? If there is an error, please post that, and be specific about the steps and results.

3. How are you starting FF? This sounds more like a FF issue, but could be anything. Again, please be specific.

4. Have you isolated the failing steps? Is there something in the test above this step that is causing a change in the structure? Just because Spy sees it doesn't mean that it is valid at test run. Have you tried turning off caching in the repository, for example?

5. Again, this sounds like the object being validated is no longer valid due to caching.

Caching takes the object, the FIRST time it is seen by Ranorex, and tries to find that same object every time. If the structure of that object, or any of its parents, changes, it will fail to find it. If caching is off, Ranorex re-evaluates the RXPath every time it encounters the object (NOTE: this can cause lag and slowness, and should be used sparingly).

Re: Inconsistent Playback

Posted: Fri Oct 11, 2013 3:32 pm
by Mozzytm
Unfortunately this is one of those projects where i cant really share anything. i will try disabling caching and the clicking in text entry (fairly sure i do this one anyway but i will double check). Its really tips like this i was looking for to increase stability. As i say test run #1 generally fine, 2-3-4....100 that's anyone's guess whats going to happen. Consequently even though the process is automated i still have to sit zombified in front of the screen ensuring it completes.

What i may do is Automate a combination of windows Calc and MSN.com or something to get the issues replicated and post that up here.

Oh and my firefox starting is invoked by the "Open Browser" action in the rxrec with the Url and Browser set to my test app and Firefox respectively

Re: Inconsistent Playback

Posted: Mon Oct 14, 2013 2:44 pm
by mohsenish
Hello;
I agree with Mozzytm. Sometimes Rx playback doesn't play well and we must sit in front of screen and watch if Rx works right. For example i have a mouse click in my scenario and playback it with Rx. If i play it once it is Ok but when i play it twice it goes wrong and doesn't click on the button. I am really amazed that what goes wrong? In some cases this issue can resolve with have two mouse clicks instead of one mouse click but in other cases i haven't any idea.
But i steel think that Rx is Great. :D

Re: Inconsistent Playback

Posted: Tue Oct 15, 2013 4:58 pm
by Support Team
Hello mohsenish,

This sounds like a timing issue. Please try to insert a delay before you click the button.
Which error message do you get? Can you please check if the Ranorex Path is correct for the button?

Regards,
Bernhard

Re: Inconsistent Playback

Posted: Sat Oct 19, 2013 6:27 am
by mohsenish
Hello Bernhard;

Thanks for your reply;
I think this problem isn't for timing issue.I try insert delay before click action but still it doesn't work. My Ranorex Path is also true. In this case when Ranorex Play back scenario, mouse click command complete but it doesn't click on the button and Ranorex wait for next command until timeout.
In this link you can see a video file that represent what i said about inconsistent playback. In this video you can see ,when Ranorex playback my scenario (click on the user icon); user icon move down and up but i just click once on the user icon. This problem Causes other steps goes wrong. :(

Re: Inconsistent Playback

Posted: Tue Oct 22, 2013 4:09 pm
by mohsenish
Hey there,
I found a solution for my click issue,i use VNC remote control instead of windows remote control.:D :D
I play a scenario 20 time and all of them are successful. :o :o

Re: Inconsistent Playback

Posted: Tue Oct 22, 2013 5:05 pm
by Support Team
Hello mohsenish,

Thank you for the information.
I am glad that you could solve the issue.

Regards,
Bernhard