Posted by Tobias Walter on Thursday, October 18th, 2012 at 10:01 am to Integration

Recently, the more commonly used method for developing software has been shifting from traditional models like the waterfall or the v model to a more iterative model: the agile software development.

Agile development processes are implemented using continuous integration tools. To keep individual iterations speedy, it’s absolutely essential to bring automated testing into the continuous integration process… and likewise into the agile development process.

Ranorex can easily be integrated into continuous integration systems as Ranorex Studio creates an executable from of its Test Suite projects that can be triggered by almost any CI system.

This blog post explains how to integrate Ranorex automation into your existing continuous integration process.

CI Integration

How it should work

The principle behind CI approach is to frequently promote code changes and rapidly get feedback about the impact these changes have on the existing system.
So basically every time a developer or a tester commits changes to the version control system, the version control system should inform the CI system that there are changes to the system under test.

This information triggers the CI system to build the application under test as well as the Ranorex automation project if necessary and executes the Ranorex automation project.

To evaluate the outcome of the test, the CI system usually examines the return value of the executable or its output text (e.g. “TEST FAILED” for failure). With Ranorex the return value ’0′ signals the successful execution of the test script and the return value ‘-1′ signals a failure.

Each involved team member should get notified about a finished build. The notification should include build logs as well as reports of the executed automation.

A closer look at the CI System

Let’s have a closer look at the CI System.

CI System

Note: It is necessary to have the Ranorex main components – and a valid Ranorex license – installed on each machine on which you are going to execute Ranorex code.

The whole build process will be triggered by committing changes to the version control system. For this reason the source repository as well as a mechanism for checking for changes (e.g. a checking interval) in the repository has to be configured.

After checking out the committed code from your version control system three steps in your build process have to be added.
One for building the application under test, one for building the Ranorex automation project and one for executing the Ranorex automation project.

Build Application Under Test

The first step should generate an executable of you application under test which will later be triggered from the Ranorex Test Suite project.
For this reason add a build step which will build your AUT (e.g. MSBuild build step, Ant build step, …).

Build Ranorex Test Suite

The second step should generate the executable automating your application under test. To do so, add a build step (MSBuild or Visual Studio) and choose the project file (*.csproj) of your Ranorex project which should be built.

Execute Ranorex Test Suite

The third step should execute the just created executable. Therefore simply add an execution step triggering the *.exe file and define some command line arguments if needed.

Basically, the test execution can be triggered on the same system as the projects were built. If the execution should be triggered on another system, make sure to deploy the built executable and all necessary files to that system.
Also make sure that the executing process is able to execute the AUT and the Ranorex Test Suite in a desktop session and not in a console session (don’t start as service).

With frequently applying code changes, it does not make sense to run the whole test suite with all it’s test cases but a subset of test cases regarding the affected area. For this reason, you can use so called run configurations which can be added and edited directly in the test suite editor (as you can see in the section ‘Running a Test Suite’ in our user guide).
The following example will illustrate how a run configuration can be triggered using command line arguments:

Executing following command line will run the test suite executable called ‘TestCIProject’, run the run configuration called ‘SmokeTest’ and generate a zipped report file called ‘Report.rxzlog’ stored in a folder called ‘/Reports/’:

TestCIProject.exe /rc:SmokeTest /zr /zrf:Reports/Report.rxzlog

You can find a list of all available command line arguments in the section ‘Running Tests without Ranorex Studio‘ in our user guide.

Provide Feedback

After finishing all build and execution steps, the responsible developers and tester should be informed about the success of the builds as well as the outcome of the test execution. To do so, add a post build action which sends a mail having the build log of the build steps as well as the generated zipped report file attached.

Blog posts concerning integration into specific systems

Have a look at the following blog posts which explain how to integrate into some specific systems in detail:

Conclusion

As you can see, it’s easy to integrate Ranorex Automation in any system, simply by triggering the system to execute the Ranorex executable.
Please feel free to share your thoughts on this topic in the comments section.

Tobias Walter

Online Marketing and Content Specialist at Ranorex
Tobias Walter is Online Marketing and Content Specialist at Ranorex GmbH. Next to his marketing activities he also writes articles published in severals blogs and journals.

Latest posts by Tobias Walter (see all)

Share

Tags: , , , , , , , , , , , , , , ,

8 comments

  1. alcos

    Hey,

    Thanks for the article. I have one question with respect CI for iOS mobile. Here is my scenario.

    1. We get nightly builds of an iOS app
    2. To test this app with Ranorex we have to instrument it in xcode i.e. MacBook machine
    3. Then we need to provision the app from xcode onto an iOS device e.g. iPhone
    4. After than we can kick off Ranorex tests (Ranorex only runs from Windows machines as far as I know).

    My question is… in steps 2-3 we need to have iPhone connected to MacBook machine. Then in step 4 we need to connect that iPhone to the Windows machine running Ranorex. How do we implement continuous integration in that case? Is my understanding of the process wrong?

    Thanks in advance for your answer.
    Thanks

  2. twalter

    @alcos

    Yes, the iOS app has to be built on a mac. We are working on a possibility to deploy the app from a windows machine, but for now you have to also deploy it from the Mac. The Ranorex test are going to be executed on a Windows machine.
    The deployment can be done via WiFi as well as the testing.
    Which means you do not have to physically switch the USB connection from the Mac to the Windows machine.
    For further details please have a look at the regarding section in our user guide.

  3. Challenges in Mobile Automation Testing | Mobile Application Testing

    [...] Your mobile test automation can easily be integrated into continuous integration systems with Ranorex as Ranorex Studio creates an executable of its Test Suite projects that can be triggered by almost any CI system. To learn more about integrating Ranorex into different CI environments, have a look at the article “Integrate Ranorex into Any Continuous Integration Process”. [...]

  4. gilazo

    WE are trying to build codedui tests that reference the ranorex dlls. We are not executing our tests and build time, but wish to execute our tests on a machine that has a runtime license.

    Does our build machine have to have a ranorex license in that scenario?

  5. twalter

    @gilazo: only the executing machine needs a valid license.

  6. Valeriu

    Hello TWalter, my question about integration is, are there any ways to integrate Ranorex with TestRail, so we’ll write & run the automated test from within TestRail?!

  7. Tobias Walter

    @Valeriu
    I do not have any experience with Test Rail, but basically you can integrate Ranorex in any tool as long as the tool can execute *.exe or *.bat files and fetch and react on the given return value. Simply check if the mentioned requirement is covered with TestRail.

  8. Make your Ranorex Report comprehensible for Jenkins | Ranorex Blog

    [...] Blog post on how to integrate Ranorex into any CI tool: http://www.ranorex.com/blog/integrate-ranorex-in-any-continuous-integration-process [...]

Leave a Reply