How to structure a solution for a huge company

[BS_MR]

Hello, I work in a big company, and we are starting using Ranorex for automatize some manual tests. At the moment, only 2-3 people of the same team are working together testing a module of web ERP Solution. So are starting thinking big(dozens of users designing their hundreds test cases at the same time), but we need advice from more experimented people to face it properly. So I will explain how are we currently working, based on the issues we had and other conflicts we faced with concurrency and git, and then you can give us your advice to improve our procedures.

We are using GIT for version control. And we will execute all the test cases in a virtual machine every day with Jenkins. So we have one big solution, inside this solution we got several projects:

Solution 1
|------------Sandbox project
|------------TestCase1 project
|------------TestCase2 project
|------------TestCase3 project
...

So as you can see, we have a master sandbox project where we execute all test cases and where we try to merge all other projects repositories. Every time we are adding a test case we create a new project in a new branch of git. In this new project, we copy the sandbox repository to reuse previous items tracked. We create records in these projects if we cannot reuse anyone of the sandbox project. Once it is finished, we push it and then merge to the master branch in git. Then the git supervisor, will take all of these records, and copy to the sandbox project, merging the associated repositories. Then he will clean up the project and push the changes in the master. The supervisor will pull the new changes in to the virtual machine. Then others can merge the new changes to their actual branch and keep working with the new changes, or start a new branch with the last changes.

In the other hand, we pretend to use Jenkins to program run the sandbox test suite once a day. That test suite will contain all test cases of other projects, but sometimes we want to execute only some of them. Can we do it directly with Jenkins?

The solution at the moment contains 30 different projects with hundreds of tracked items, and start taking a long tine to build he solution.

So now it's your time, if you have reach this point, thank you for reading all. Are we structuring Ranorex properly? Will you change anything to make the process more efficient? Any suggestions?

Thanks in advance,
Miguel

[qwertzu]

hi,

This seems to be a very structured and good approach in working with git for avoiding merging conflicts.

regarding your question:

In the other hand, we pretend to use Jenkins to program run the sandbox test suite once a day. That test suite will contain all test cases of other projects, but sometimes we want to execute only some of them. Can we do it directly with Jenkins?

Via command line you can trigger specific test cases from a project.
So in your case, you could run a batch file from Jenkins which contains commands for triggering test cases.
The exact commandline arguments can be found here: https://www.ranorex.com/help/latest/les ... ommandLine

regards, quertzu

[BS_MR]

Ok, glad to hear it. Thank you qwertzu for your feedback and also for the link.

If anyone wants to add something will be welcome

[jan]

Hello,
this solution seems to be very similar to ours. I would like to ask you for advice. It is not about git, but it is related to big solution and many test cases.

You have many test cases, you are testing many screens. How did you solve test continuance problem, when some test fails on some particular screen? For example, you are testing some detail screen and error occurs. Now, Ranorex output the mistake and moves to another test, but the app remains open on that failed screen. And next test case doesnt start on that screen, it starts on another screen. So the test fails because ranorex could find the initial UI element. And this continues to the end.

What is your solution?

Thank you very much for any advice.
Regards.
Jan

[Stub]

I personally try to write tests that clean up after themselves so other tests can continue. Or at least clean-up what they can very reasonably anticipate might occur. It's not always possible and I simply don't tend to go to extreme lengths to resolve any and all errors. If it's a problem that recurs then I see if I can restructure things a bit to cope better, especially if it's related to what I'm testing. I can often get away with closing the active 'document' - the data we're working on - and that can clean up a lot, though sometimes it has to back up through several parent levels to get to a continuance point - using the Ranorex Continue On Error settings at various levels*.

But if it's completely unanticipated and borks everything then I'll either manually disable the testing in that area until somebody fixes whatever has caused the issue, or simply wait until they do so.

* TBH, I'm still fine-tuning my own strategy here, as we build up our library of Ranorex tests. I am spreading out across more Test Suites nowadays though.

[jan]

Hello,

thank you for quick answer. However, it doesn help me much.

Unanticipated alert (for example) will still block Ranorex from continuing in test suite. And no matter how small the suite is, that alert will still block everything else .

AUT has to be brought back to some default screen. And the only way I heard off is to switch AUT completely and start it again. After every Test case. This switching of and turning on has to be conditioned by the result of previous test. And these conditions must be custom written - that is what bothers me, I aint programmer and the API documentation is not ideal, even Ranorex admits it (improved version should come out soon, but I cant wait for it).

I thought this kind of problem must solve every tester, especially since automated tests run on servers often during night, there is no way, that someone can manually adjust the flow of the tests (and that would also denied the purpose of automated testing).

I will try to come up with something after I spent sometime researching.

Thank you anyway.
Jan