Page 1 of 1

Evaluating Ranorex for our company

Posted: Mon Jan 30, 2012 9:56 am
by Stian
Hi there!

I am currently evaluating Ranorex for use with the UI automation scheme in our company and have a few questions.

Our development environment is Microsoft-based with heavy use of Visual Studio 2010, TFS 2010 and Microsoft Test Manager and we have numerous products we have recently started implementing automated UI tests for.

One of our products is a fairly big .NET website using a lot of Flex, and this is what led me to Ranorex since VS/MTM doesn't support Flex.

So far, I've managed to run Ranorex tests on Flex from within Visual Studio and it is looking very promising.

Now, to the questions:

1) Right now, I am running the Ranorex .exe recording from my test within Visual Studio like this:

Code: Select all

 try
            {
                RanorexTest.Recording2.Start();
            }
            catch (Ranorex.ValidationException)
            {
                Assert.Fail();
            }
All I do is make a reference to the .exe file and the Ranorex.Core assembly within Visual Studio.

Is this a recommended way of doing it?

2) Will the above way of running the Ranorex tests enable me to generate TFS reports where the Ranorex tests are included?

3) Are there any pitfalls I should be aware of when using Ranorex in combination with VS/MTM?

4) With VS Feature Pack 2 installed, I can set things up so that the automated UI tests can be run (after an automated build) against both Internet Explorer and Firefox.
Using Ranorex, is it possible to set the tests up to be run against f.ex. Chrome and Safari?

Hmm, that's all I can think of for now.

Anything else I should take into consideration while evaluating Ranorex? What other benefits might we gain from investing in it?

Stian

Re: Evaluating Ranorex for our company

Posted: Mon Jan 30, 2012 2:23 pm
by Support Team
Hi,

Add 1:
The recommended way to start a Ranorex module is to run it via the TestModuleRunner:
TestModuleRunner.Run(YourRecording.Instance);
So your Ranorex project is a "Console Application" and you created your Recordings and UserCode files with Ranorex Studio and you want to execute them with Visual Studio?
Did you know that you can also generate a "Class Library" out of your Ranorex Solution?
Just right click on your Ranroex project and select "Class Library" instead of "Console Application" as "Output Type".

Add 2:
What do you exactly mean with "where the Ranorex tests are inlcuded"?
What do you want to accomplish? Do you want to include the Ranorex reports into the TFS reports or do you just want to add specific log files to it, or ...?

Add 3:
I am not aware of some specific pitfalls, because we have not much experiences with MTM, but there are two blogs which cover the topic how to use TFS and VS with Ranorex.

Add 4:
Yes it is possible to run against FF, IE, Chrome, Safari.
Normally I would suggest using repository variables which can be set via the data connector/data binding to a specific browser dependent value, but because you just use the exe or the dll of the Ranorex project you are not able to set a repository variable, so you have to manually edit the RxPath or the repository if you want to run the same test on a different browser. You can also create Recordings which doesn't use the repository, but which use the RxPath to create appropriate adapters or to search for your elements like:
Button bu;
Host.Local.TryFindSingle("/button[@text='"yourModuleVariable"']", out bubu);
or
Button bu = "/button[@text='Start']";
Regards,
Markus
Ranorex Support Team

Re: Evaluating Ranorex for our company

Posted: Mon Jan 30, 2012 6:47 pm
by Stian
Hi there Markus and thanks for your helpful response!

Point 1:
Aha, I'll give it a go using TestModuleRunner...

What is the benefit of using a class library instead of a .exe?

Point 2:
What I want to accomplish is include the Ranorex test run results into the TFS reports. Some of my automatic UI tests are made in MTM/VS and some are made in Ranorex - my plan so far is to run everything as if I created them in Visual Studio and make reports in TFS covering all of them.

Point 3:
Thanks, I'll have a look :)

Point 4:
Okay, this sounds really great! I need to figure out how to do this.
If I can make a UI test in Ranorex recording from Internet Explorer and then run this same test against Firefox, Safari and Chrome that would be a very convincing argument for us to purchase your product!

Another question:
My initial plan was to create tests in Ranorex and make a unit test in VS for each of the Ranorex tets. This way I can run all the tests as tests inside VS, hook all of them up to Test Cases and even add them to test plans and test suits in Microsoft Test Manager. This should make it easy to create test reports in TFS.

Does this sound sensible to you? Maybe there is a more efficient way?
I have not considered using Test Suits in Ranorex - so far I am just making "loose" tests in Ranorex and adding them to suits in MTM. It is kind of tempting to skip VS and MTM altogether, but I need to hook the tests up with TFS work items.

Re: Evaluating Ranorex for our company

Posted: Tue Jan 31, 2012 2:06 pm
by Support Team
Hi,

You're welcome!

Add 1:
There aren't big differences, but if you just want to reference to a Ranorex project there is no need to create an executable and it is not the preferred object orientated way.

Add 2:
Do you want to just collect the different report files or do you want to save all your results to one custom created TFS report?

Add 4:
Yes you can make it this way, or as you have described in the following post, you can also create all your test scripts with VS, the Ranorex repository and the Ranorex API. I think this is the more flexible and better solution.
Regarding the question "how I can setup a browser independent test", I already replied to the following post.

Kind Regards,
Markus
Ranorex Support Team

Re: Evaluating Ranorex for our company

Posted: Tue Jan 31, 2012 2:23 pm
by Stian
Thanks again, Markus!

I am getting stuff set up and working pretty good so far. Creating the tests in VS through the Ranorex API seems like a pretty good way to go about it. I think I'll miss the recording, though :)

Regarding point 2:
I am not so sure anymore and I don't need to figure it out yet either. I know that it is possible to get the reports from the Ranorex tests into TFS and that's good enough for me.