Creating a new test suite?

Ask general questions here.
tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Creating a new test suite?

Post by tallahassee101 » Tue Mar 29, 2011 5:08 pm

Is it possible to create a new test suite without creating a new solution? I would like to have multiple test suites within my solution and do not want to have multiple solutions.

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Creating a new test suite?

Post by Support Team » Wed Mar 30, 2011 9:55 am

Hi,

you can add another Test Suite by simply adding a new project to your solution.
then you have a Test Suite for each of your projects in one solution.

Or you only have one Test Suite with one folder for every test suite you'd like to have.
Let's name the first folder "Test Suite 1", the second one "Test Suite 2" and so on.
now you can make a testconfig for each of these Test Suite folders to run only the expected folder.

Kind regards,
Tobias
Support Team
.
Image

User avatar
Gunner1980
Posts: 89
Joined: Mon Apr 05, 2010 8:44 pm
Location: Austin, Texas

Re: Creating a new test suite?

Post by Gunner1980 » Wed Mar 30, 2011 4:03 pm

So you have to create a new project and then create a test suite under that project? Is that the correct process to create multiple test suites under one solution?

I tried this but when I create a new project it does not include a .rxtst file by default. I had to copy it from an existing project and rename it.

Am I missing a step here?

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Creating a new test suite?

Post by Support Team » Thu Mar 31, 2011 9:13 am

Hi,

We recommend to use only one Test Suite per solution. Is it really needed to use two Test Suites in one solution? Can you explain a use case where this is required?
We have configurations for the test suite, where you can select different sets of test case which should performed.

Regards,
Peter
Ranorex Team
.
Image

User avatar
Gunner1980
Posts: 89
Joined: Mon Apr 05, 2010 8:44 pm
Location: Austin, Texas

Re: Creating a new test suite?

Post by Gunner1980 » Thu Mar 31, 2011 5:05 pm

Yes there is totally a need for multiple test suites per solution. I don't want one exe that has 5000 tests in it. I want them modular so there is an exe that tests each functional area. We have to consider that users run our tests that are not developers at our company. I am developing test cases for a very intricate system that has many different functional areas. I also have non developers using my test cases so in order to simplify the process and to keep the exe from being gargantuan I need different test suites. What if I had 5000 tests and I only wanted to run one? I would have to scroll through 4999 items since there is no expand all/collapse all option in the Ranorex test suite.

I want to be able to share my libraries and repositories between test executables as well which is why I need multiple test suites. If I have to have a separate solution for each test suite that means I can't share repositories and libraries.

Additionally we asked Ron when he came out here for training how to best perform what we wanted to do and this was his recommendation. When you say you recommend only one test suite per solution does that mean that multiples haven't been tested? Is this something that Ranorex will support?

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Creating a new test suite?

Post by Ciege » Thu Mar 31, 2011 5:21 pm

I may be off base here (since I don't use test suites) so forgive me if I am...

One thing you might consider is in your solution create a form that pops up on launch that has check boxes or radio buttons (etc...) that list each test that is in the solution. The end user can select which test or tests they want to run. Then you can have code that, depending on which test(s) is/are selected run only that/those tests. Basically, you would be creating a launcher of sorts that shows all the tests available and allow for selection of which test or group of tests to run at any given time...
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

User avatar
Gunner1980
Posts: 89
Joined: Mon Apr 05, 2010 8:44 pm
Location: Austin, Texas

Re: Creating a new test suite?

Post by Gunner1980 » Thu Mar 31, 2011 6:31 pm

That is what the 3.0 test suite has available Ceige, a bunch of check boxes to select what tests you want to run, but I have different hardware product lines that I want to have their own exe's. They share the same software so I want to be able to share projects and recordings but I don't want them all in one exe. I am to the point where I could have hundreds of test cases and this would be very cluttered for a user to go through each one and check/uncheck boxes, especially for tests that are for another product or that are in a completely different functional area.

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Creating a new test suite?

Post by Ciege » Thu Mar 31, 2011 6:39 pm

Again just throwing ideas out to see what might stick...

Instead of using the built-in "launcher" dialog, crate your own that has a drop down that the user picks the product line being tested. Based on that selection give a selection of tests available for the user to execute. So this possible solution only shows what tests are available for the user based on the product being tested.

Another is to create a backend, shared framework. Compile that as a DLL. Then create new projects for each product line that references the shared DLL. This way you have one location for all your shared code (the DLL). You only need to make changes/fixes/additions in one place and all the separate projects always have the latest code. Each separate project can then have only the code that is specific to them, and in turn creates a specific EXE for just that project.

Again, just ideas...
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

User avatar
Gunner1980
Posts: 89
Joined: Mon Apr 05, 2010 8:44 pm
Location: Austin, Texas

Re: Creating a new test suite?

Post by Gunner1980 » Thu Mar 31, 2011 7:29 pm

Ciege wrote:Another is to create a backend, shared framework. Compile that as a DLL. Then create new projects for each product line that references the shared DLL. This way you have one location for all your shared code (the DLL). You only need to make changes/fixes/additions in one place and all the separate projects always have the latest code. Each separate project can then have only the code that is specific to them, and in turn creates a specific EXE for just that project.

Again, just ideas...
That is exactly what we are doing. But if Ranorex recommends only having one test suite per solution that would only create one exe which is why I am asking the questions that I am. Is there a way to create multiple exe's using one test suite in a solution? The only way I could figure out to make individual exe's was to create multiple test suites. Don't get me wrong I love the functionality the test suites give us, I just want to have multiples broken out by product that share the same DLL's. Thanks as always for the input Ciege.

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Creating a new test suite?

Post by Ciege » Thu Mar 31, 2011 7:51 pm

Ah, light dawns over marble head... Sorry for being dense, but I guess that's what I get trying to answer questions in a scenario I don't work in...

I do now see your point, I will kindly bow out and see what support can do for you.

Good luck!
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Creating a new test suite?

Post by Support Team » Thu Mar 31, 2011 11:14 pm

Gunner1980 wrote:When you say you recommend only one test suite per solution does that mean that multiples haven't been tested?
It has been tested, but not as thoroughly as solutions with only one test suite. Additionally, only one test suite can be active at a time, the one that is contained in the start-up project of the solution (you can set the start-up project by right-clicking on the project and choosing the corresponding item in the context menu).
Gunner1980 wrote:So you have to create a new project and then create a test suite under that project? Is that the correct process to create multiple test suites under one solution?
Yes, as a test suite project can only contain a single test suite, you have to add a new test suite project for every test suite you want to add to your solution.
Gunner1980 wrote:I want to be able to share my libraries and repositories between test executables as well which is why I need multiple test suites. If I have to have a separate solution for each test suite that means I can't share repositories and libraries.
This is not really true. Even if you have separate solutions, you can still have common repositories, test modules and libraries. The recommended way is to have such common modules in a separate project. Then you can add that project to every solution that needs the common functionality.

Basically, there are two alternatives for creating multiple test suites that use common functionality: the first is to have one big solution containing the common module projects and a test suite project for every test suite. The second is having one solution per test suite (with only one test suite project in it) and referencing (either as DLL reference or by adding the project to the solution) the common module projects to the solution.

The choice is up to you. My personal preference is having multiple solutions, since otherwise solutions with multiple test suite projects and common module projects in them might get really big...

Regards,
Alex
Ranorex Team
.
Image

User avatar
Gunner1980
Posts: 89
Joined: Mon Apr 05, 2010 8:44 pm
Location: Austin, Texas

Re: Creating a new test suite?

Post by Gunner1980 » Tue Apr 05, 2011 4:24 pm

But if you have multiple solutions and you are adding the same library/repositories to multiple solutions how can you tell if a change breaks another solution?

If you have all the test runners included in your solution you can build the solution to ensure any changes you made did not affect the other test runners.


Am I correct in saying the following? That if I have separate solutions one solution may build with the changes you have made to a shared library/repository but another solution may not? Would I have to open each solution I have sharing that library/repository and build them individually to ensure the changes I made to the library/repository didn't break one of my other solutions?

That seems much more labor intensive than having multiple test runners in one solution to guarantee my changes did not break my other test runners.

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Creating a new test suite?

Post by Support Team » Wed Apr 06, 2011 10:02 am

Hi,

it's your choice how you get things done.
But as Alex mentioned before, the second approach is the recommended one.
When your solution is getting really big, that might lead to performance issues which won't occur on small solutions.

But as said before you have to decide by yourself.

Regards,
Tobias
Support Team
.
Image

User avatar
Gunner1980
Posts: 89
Joined: Mon Apr 05, 2010 8:44 pm
Location: Austin, Texas

Re: Creating a new test suite?

Post by Gunner1980 » Wed Apr 06, 2011 4:24 pm

Support Team wrote:Hi,

it's your choice how you get things done.
But as Alex mentioned before, the second approach is the recommended one.
When your solution is getting really big, that might lead to performance issues which won't occur on small solutions.

But as said before you have to decide by yourself.

Regards,
Tobias
Support Team
If it is the recommended approach please tell me how I am supposed to tell if my libraries will work between multiple solutions. The only way I know how to do this would be to open each solution individually and try to build it. This seems like a lengthy process. What benefits do I get if I use this approach?
Last edited by Gunner1980 on Wed Apr 06, 2011 5:57 pm, edited 1 time in total.

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Creating a new test suite?

Post by Ciege » Wed Apr 06, 2011 4:28 pm

You could write a simple build script that builds all of your projects automatically and exports the results to a log file. You could even have the build script parse the log file results and report on the pass fail status at the end of the batch.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...