Ranorex Comparison Info

Experiences, small talk, and other automation gossip.
acates
Posts: 8
Joined: Tue Nov 16, 2010 8:10 pm

Ranorex Comparison Info

Post by acates » Thu Dec 02, 2010 5:54 pm

Hello,

My company is in the process of evaluating the Ranorex product to make a purchasing decision. One of the members of the IT team has raised the question about a scripting tool called AutoIt (http://www.autoitscript.com/autoit3/index.shtml).

I would appreciate any help to address the following questions:
  • 1. Does Ranorex have a document comparing its product features / benefits to competitors like AutoIt?

    2. Does anyone have any hands-on experience with the Recorders in Ranorex and AutoIt, and how do they compare (e.g. ease of use, do they both generate code, etc.)…?

    3. Most of our developers / testers are comfortable using VB.net so we plan to use this language with our Ranorex recordings / coding. How does the AutoIt “BASIC-like scripting language” compare from a learning curve, ease of use, user-base support, etc…?

    4. We are planning to test both Windows/GUI apps, custom ASP Web apps, and a few ERP packages like Lawson (web version). How do the object identification capabilities compare between Ranorex and AutoIt…?

    5. The final question that I am being asked is, why should my company pay for Ranorex when AutoIt is free…?
If I have missed any questions / factors that should be considered when comparing products, please let me know (I'm open to suggestions). Thanks in advance for your help to answer these questions as we finalize our purchasing decision…!

Cheers,

AC

User avatar
Ciege
Posts: 1336
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Ranorex Comparison Info

Post by Ciege » Thu Dec 02, 2010 6:33 pm

Hi AC,

I use AutoIt all the time... It certainly has its place in my "tool belt" along with a *real* test automation tool like Ranorex. Much like any profession, there is no one tool that does it all.

One of the biggest projects I used AutoIt with at my current job was to create an automated build environment to build our software here. Quite elegant, works fine and (as you said) it was free. However, AutoIt is not my tool of choice for doing "test automation". The AutoIt recognition methods just do not work well for much of my AUT. Especially when I want to try and do some of the more intricate tests with 3rd party components. If all you are going to be testing on is standard IE and standard Windows components you might be able to get away with using AutoIt.

Another detractor for me with AutoIt and true test automation is AutoIt is *only* single threaded. My test automation using Ranorex in Visual Studio runs multiple threads during testing. One thread is my main test thread and at least one other thread watching for unexpected exceptions or AUT crashes. Sometimes I also have other threads watching performance of the CPU, memory, disk caching, etc...

Finally, in the grand scheme of things, Ranorex is very inexpensive for what you get. Ranorex is designed as a test automation tool while AutoIt is designed as an automated scripting utility. Similar? Yes. But designed to do different jobs. Can you hammer in a nail with a screw driver? Sure but do you really want to?

My best suggestion would be to have whomever is doing the Ranorex evaluation at your company also download AutoIt. Have them write a test script for the exact same test once in Ranorex and once in AutoIt. Make sure this is a true to life test that will expose the strengths and weaknesses in each method. Also make sure that you test against various different objects within your AUT(s). You need to make sure that each tool can access your objects. Use that information to help your decision. Remember, free is great as long as the tool does what you need it to.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

User avatar
slavikf
Posts: 104
Joined: Mon Sep 13, 2010 9:07 pm
Location: Toronto, Canada
Contact:

AutoIt vs Ranorex

Post by slavikf » Thu Dec 02, 2010 9:09 pm

AutoIt vs Ranorex? I used AutoIt before. Good tool, but has some limitations:
2... Does anyone have any hands-on experience with the Recorders in Ranorex and AutoIt, and how do they compare (e.g. ease of use, do they both generate code, etc.)…?
AutoIt has no Recording capability. All code has to be written manually.
3. Most of our developers / testers are comfortable using VB.net so we plan to use this language with our Ranorex recordings / coding. How does the AutoIt “BASIC-like scripting language” compare from a learning curve, ease of use, user-base support, etc…?
Ranorex uses "real" or standard languages: C#, VB.NET and Python (IronPython), which many people already familiar with. AutoIt using "BASIC-like scripting language designed for automating the Windows GUI and general scripting", which is not complicated at all, but need to be learned.
Also, that mean, they differently use external libraries.
Ranorex:
  • Can use any existing .NET libraries (including .NET framework libraries).
  • Can use libraries and PlugIns, published here by Ranorex team
AutoIt:
  • Directly call external DLLs and Windows API functions (that's not as easy)
  • Can use libraries designed for AutoIt (they have quite few published on their website)
4. We are planning to test both Windows/GUI apps, custom ASP Web apps, and a few ERP packages like Lawson (web version). How do the object identification capabilities compare between Ranorex and AutoIt…?
AutoIT works with Windows GUI only. Can't test Web apps.

Also, objects recognition in RanoRex is MUCH better, than in AutoIt. And not only in sense - "if AutoIt can or can't recognize object", but also the way, HOW it recognize it. I had issues in AutoIt, when object changed it ControlName depending on how many times it's window was displayed. I.e. - It had one name, when app was just started and another, after i reopened it's window.

All that said, AutoIt is good tool, and it certainly good fit for some projects. It just can't handle all tasks, which Ranorex can.

User avatar
Ciege
Posts: 1336
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Ranorex Comparison Info

Post by Ciege » Thu Dec 02, 2010 10:57 pm

slavikf, unfortunately a couple of your points are incorrect:
AutoIt has no Recording capability. All code has to be written manually.
And
AutoIT works with Windows GUI only. Can't test Web apps.
Both of these points are incorrect. AutoIt does indeed have a recording capability when you download the SciTE package. You can record actions and it will generate code for you. Also, you can do plenty of web testing through the DOM, much the same way that Ranorex (and other automation utilities) work through the DOM.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

User avatar
artur_gadomski
Posts: 207
Joined: Mon Jul 19, 2010 6:55 am
Location: Copenhagen, Denmark
Contact:

Re: Ranorex Comparison Info

Post by artur_gadomski » Mon Dec 06, 2010 8:35 am

I am going to agree with Ciege on the point about trying them both out. I chose Ranorex for my company because automating the same test took me 1 day in C# in Ranorex and 1 week in JScript in TestComplete.
While you're at it you might just as well evaluate a couple of other tools. 28% of tools become shelfware so it's important to make a well informed decision before you spend money.