Ranorex Forum

Hi,

We are about 4 months into our overall automation efforts; we have started the effort using Ranorex. We are considering using the 1 solution / many projects approach instead of mostly separating test suites in their own solution that references an auto framework library. We are using DevOps GIT for our source control and run some test suites using DevOps Pipelines.

Having read through the forums, I see many are using the single solution approach. Seems run configurations are a good way to setup project runs within Studio so that you can test while building the test. Some posted having many or a lot of projects along with a framework lib. I want to get more specifics. I appreciate the help!

1. How many projects have you reasonably managed in a single solution? Was there a project count threshold where it had to be split? In your setup, does each project have only 1 test suite?

2. Do you commit the solution to a single branch in your source control? Do you version your test suites for the various releases of your app? Please elaborate with your setup.

3. Were there any gotchas encountered as your automation effort has grown?

4. If you're using DevOps Pipelines, how are you running a single project from the solution? Can you run 2 projects back to back from a pipeline task?

5. What's your team size?

Thanks!

1. Our test solution has 7 projects in it, though one is just for experimentation from when I was first learning Ranorex. We also have 7 Test Suites in just one of those projects - so the bulk of our projects are framework/library projects utilised by the master project containing all of our Test Suites.

I am not personally concerned with lots of projects, as our products under test are vastly larger. Keeping everything in the one solution is more important to me. Structuring that solution and the contents of each project is critical to finding stuff.

I split the projects on a logical basis. Sometimes I correctly anticipate the growth, other times I have to split things into a new project during development.

2. We have branched versions of the Ranorex solution for older released versions of our software. This matches how we deal with different versions of our product moving forwards, so it comfortably fits into that model. Once branched, both the product and tests undergo only essential maintenance - a snapshot in time that gradually becomes ever more static as it recedes into the past.

3. Support for multiple Test Suites in Ranorex, and thus our splitting into multiple Test Suites, has been a game changer here. That fundamentally altered how I built things. I then combined Run Configurations to support different product flavours. Packing too much into a Test Suite and knowing when to make a break is important to recognise - the earlier the better IMO. I'm more likely to consider test structure for the longer term now.

4. IIRC we execute multiple Test Suites sequentially from our build pipeline, then combine the test reports into our notification system.

5. It's just me building our tests. Almost nobody else here has any interest in them! A couple have talked about it but Ranorex is coming up to its 4th birthday here so I can read the tea leaves by now I suspect...