Page 1 of 1

Tool for Distributed Execution of Tests?

Posted: Tue Mar 10, 2015 8:50 pm
by tgagel
Hi.

We have a couple of floating runtime licenses and have Ranorex installed on a couple of Remote Desktop Machines to allow our test group to run tests while still using their local machines for other tasks. Currently we have to set up each tester on each remote machine and they have to know how to use the Test Suite Runner (for example, when to change the drop down, when to check/uncheck tests, etc). This has the disadvantage that if one tester changes a test suite's makeup incorrectly, it will remain in the bad state for other testers until a new version of the tests are deployed or it is manually fixed.

We'd like to streamline the way we run tests on remote machines. Are there any tools or plugins that work with Ranorex that would allow the tester to:
  • - Pick which tests to run
    - Choose parameters for the tests (e.g. which environment(s) to test)
    - Check remote machine availability & assign tests to remote machines (first come first serve)
    - As tests complete, assign next tests to the available machine
    - Show test progress to the tester
    - Report test results upon completion or as tests are completed (or point the user to the results)
Basically I'd like to make it so that our testers don't to use remote desktop and don't need to figure out which remote desktop machine may be available while also using our machines & licenses in the most efficient manner (automatically distribute next test to the next available machine).

(I've read through the 'Integrate Ranorex into Any Continuous Integration Process' blog posts and, perhaps there is something there, but I can't tell if any of the tools would support just manually launching tests without the building & deploying that seems to be the CI tool's primary purpose).

Thanks and info would be appreciated.

Re: Tool for Distributed Execution of Tests?

Posted: Tue Mar 10, 2015 9:08 pm
by odklizec
Hi,

Continuous Integration is exactly what you are looking for. True, CI tools are best for build-deploy-test scenarios. However, they can be configured in completely different way. Unfortunately, that manual selection of tests, you mentioned in your post, seems to go against the idea of continuous integration, where no user interaction is expected/required. I can imagine setting the CI job with several conditions, which run different set of tests. For example do different tests during day (smoke test after each commit) and different set of tests during night (more thorough tests). CI configuration possibilities are endless ;)

Re: Tool for Distributed Execution of Tests?

Posted: Wed Mar 11, 2015 4:21 pm
by Ciege
You can also look into using PSEXEC as the method of pushing your automated test to a remote machine and running it. I've done similar in the past (not in CI). I wrote a small app that would allow a single user the ability to check the tests that they wanted to run, compile the test(s) to an EXE, distribute the proper files to the selected test machine(s) that had a runtime license installed, launch the test EXE (via PSEXEC) and wait for the process to complete giving me the result of the test.

It's not trivial to setup but not beyond the means of a single decent developer either...

Re: Tool for Distributed Execution of Tests?

Posted: Thu Mar 12, 2015 2:59 pm
by krstcs
Either of the approaches listed would work.

I use Jenkins CI and it is pretty good for this kind of thing once you get it set up.