Hooks after certain actions

Ranorex Studio, Spy, Recorder, and Driver.
DominikR
Posts: 5
Joined: Tue Aug 05, 2014 9:15 am

Hooks after certain actions

Post by DominikR » Tue Aug 05, 2014 10:03 am

Hello!

We've recorded some evaluation tests with Ranorex. Our main problem with web tests in the past has been instability, i.e. waiting/delays/etc. resulted in non-deterministic tests. This is a major pain point.

Therefore we're looking for a new web testing framework. Using Ranorex' turbo mode, we've disabled all delays and waiting times. The search timeout should be the only "timeout" value, we've set it to 10s.

However, this resulted in instability with Ranorex as well, elements are not found which are clearly visible on the screenshot, etc.

Is there anything else we can do? Are there coding "hooks" available, like the Init() method, which are called before/after each Click() where we can wait for a specific, deterministic event like "the text content of the span with id pageUpdateCounter has increased by one".

All the best,
Dominik

krstcs
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Hooks after certain actions

Post by krstcs » Tue Aug 05, 2014 1:33 pm

I would recommend that you set the timeouts back up to 30s or higher. Understand that this factor does not make the test RUN any faster, it only makes it FAIL faster. Many people don't realize that. Ranorex will find the objects as fast as it can, UP TO the timeout. If it hits the timeout it will fail.

Even though computers (and Ranorex) are very fast, they still require time to work. While automation is, in general, faster than a human, that doesn't mean that it can, or should, be run at the maximum speed possible. In Ranorex's case, it needs time to search each possible path to find the object you are looking for.

Websites (and any other software, although the web is factors slower) take time to fully load and render, and even if you can see the elements, that doesn't mean the website is ready to take action based on user input. You will still have to wait for the page to get to the "complete" state.

I recommend a few things in this situation:

1. Make all of your recording modules as small as possible. For example, if you need to click the OK button, make a module called "Click_OKButton" and only have the one mouse-click action in it. If you need to enter text in a username field, name the module "Enter_UserName" and only put the actions required to get a string of text entered in the correct field (i.e., click the field, ctrl-a, del, enter keysequence, validate text). This allows you to string the modules together in each test case to form the test flow logic.

2. Make ONE module that is your timing module and call it "_WAIT_FOR_PAGE_LOAD". (Underscore to start keeps it at the top of the list of modules for easy reference, and all caps makes it easy to spot.) Put it anywhere that the system will need to load/reload a page. All timing can be handled in this one spot. Just add one or two sets of these actions (I use 3 because our site loads a bit, says it's complete, but then loads more scripts):
1.Delay -> 1000ms
2. Wait for -> Exist -> 30s -> YourDomObject (this will wait for UP TO 30s, so it won't slow down the test)
Now you can change the timing as needed by adding more of these sets, or by just changing the number of seconds to wait.

3. Along with 2. above, add " and @state='complete'" (sans quotes, but including all spaces) to the path of your top-level webpage DOM object. This will force Ranorex to only find the DOM object when it is completely loaded.
Shortcuts usually aren't...

DominikR
Posts: 5
Joined: Tue Aug 05, 2014 9:15 am

Re: Hooks after certain actions

Post by DominikR » Tue Aug 05, 2014 1:43 pm

We have the search timeout back at 30s - I know that this is an "up to" value, but that is not our problem.
Websites (and any other software, although the web is factors slower) take time to fully load and render, and even if you can see the elements, that doesn't mean the website is ready to take action based on user input. You will still have to wait for the page to get to the "complete" state.
That's exactly my point. We don't want to wait "some time" until the page is "complete", we want to wait for a specific, deterministic event. In our case: we have some jQuery scripts which run on DOM's document.ready event, and after they have finished they produce a hidden span with an ID, which is the indicator that the page has fully loaded.

Now, we want to read the old ID, then click, and then wait for the span to have the new ID - for up to 60s waiting time, but as fast as possible.

So you say, there are no "real" hooks, we just should put a code module before and after the clicks?

krstcs
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Hooks after certain actions

Post by krstcs » Tue Aug 05, 2014 2:16 pm

If you have a specific element which you know will change, then you could wait for it to have the new value. You could grab the "old" value right before the click and then wait for the value to be different. You would need to do this in user-code though as Ranorex doesn't have the logic to handle this in the action table.

As for waiting "some time", that is how Ranorex is designed. Remember, it is a FUNCTIONAL UI, not load, response, or service, test tool (although you can rig those up in some situations).

It attempts to find the object until the timeout expires. If it finds it in 1s, great, if 2m, great (as long as you set the timeout high enough), but you have to allow it time to find the objects. If you have requirements that the object (or page) should take less than 10s, then use that as the timeout for the page and Ranorex will fail if the system exceeds that. It has to search the DOM tree (or it's cached version if nothing has changed on the page) in order to find each object.

If you need it to be faster, then the best way is to make the XPath more unique by using specific attributes, not using wildcards ("*", "//", "?"), and using Unique IDs (#'MyIdValue') as much as possible.

Also, the suggestions I posted are generic and useful for MOST situations with Ranorex. I would still recommend that you consider using the pattern I described because it will be easier to manage and understand (especially for new people) in the long run. Modules should be atomic, simple actions, and the test cases should control the flow based on how the modules are laid out.

A few questions for you (for my understanding of your situation):
Are you using Ranorex Studio, or just the Ranorex API with Visual Studio?
What test automation platform did you use before Ranorex?
How much experience do you have with .NET (or Java, C++, etc.)?


Just for your info:
I'm using Ranorex Studio almost exclusively. I've found it to be enough for what I need. It is built on SharpDevelop 3.2 so it is a pretty full featured IDE on its own.
I used QTP way back in the day, and have tried just about every other one out there at some point while evaluating them. Ranorex is the best out there for all-around automation. You might find others that are better at specific areas, or that you like better in those areas, but I haven't.
I have 15 years of experience with Java and .NET, including 3+ years with Ranorex and 4 with QTP (I hate VBScript! :D ).
I'm testing our ecommerce site, our Java-base Point-of-Sale system, some custom CAD type systems (both Java and web-based), tablet (.NET based) apps, and some other odds and ends, all with Ranorex.


Hope that helps! :D If you have any more specific questions, let me know.

Note: I don't work for Ranorex and they know the internals much better than I do so maybe they know a way to do hooking. Also, if you have a feature request (such as adding some time of before/after event hooks) they would like us to send them via email to [email protected].
Shortcuts usually aren't...