Distributed testing inquiries

Ask general questions here.
User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Distributed testing inquiries

Post by Ciege » Wed Sep 23, 2009 7:10 pm

I am looking for suggestions and comments about what people have done specifically with Ranorex (and more generically with any test automation system) to create distributed functional and load testing (i.e. sending tests out to "agent" machines to do various testing).

I know Atom says that they have developed something in house to do this so hopefully you can help with some pointers or examples.

1) What are you using to control the distributed tests (centralized master machine). How do you detemrine who gets what test.
2) How are you pushing the tests out to agent machines.
3) How do you coordinate the agents. I.e. if agent 1 needs to complete something before agent 2 how do you coordinate the two. Also, if agent 1 fails for some reason, how do you inform agent 2 and/or the master of the failure.
4) How do you centralize the reporting.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

atom
Posts: 357
Joined: Sun Dec 07, 2008 11:14 pm
Location: Dublin, Ireland

Re: Distributed testing inquiries

Post by atom » Sat Oct 03, 2009 11:41 am

Hiya

We first took a look at STAF from IBM. Its a service based framework, whereby each test machine can have "services" running on it, and can easily communicate with other machines. There is a set of standard services by default for copying files between machines, executing programs on MachineB from MachineA etc. Writing new services is darn easy in Java.

However we then decided to roll our own framework as we wanted the same distribution framework for unit tests as gui tests. We developed 3 programs; Test Controller, Test Slave, Test Logger.

- Test Controller : Receives a set of tests case .zips into an inbox, processes them to an internal queue, and then delivers the test case .zips and the AUT .zip to test machines

- Test Slave : Sits on each test machine waiting for new .zips, when received unzips them, inspects them, and executes

- Test Logger : All log file information from all test machines is sent via UDP to a central logger service

It was about 4 months development !
Looking back, Id say start with STAF !

Cheer