Bamboo CI with Ranorex Test Automation

Nov 14, 2012 | Integration

Bamboo Integration

The common method for developing software has been shifting more and more from classical development models to more agile software development. In response to numerous inquiries from Ranorex customers, we have decided to write another Ranorex integration blog. This blog post will illustrate the integration of Ranorex Automation into a Bamboo Continuous Integration Process.


The fundamental principle of agile software development is to control changes and rapidly get response about the effects of the changes.
Applied to your Continuous Integration Process the infrastructure can be sketched as follows:

Bamboo Integration

A developer or test engineer commits his changes to the version control system. Based on the committed change, the version control system triggers the continuous integration system to start a job which triggers the build agent to

  • check out the modified source from the version control system,
  • build the application under test (AUT),
  • build the Ranorex Test Suite and
  • execute the Ranorex Test Suite which tests the AUT.

The corresponding Ranorex Report file as well as a build log will be attached to the build iteration.
The developer and/or test engineer responsible for the changes will be notified about the build and test success.

Note: From Ranorex 7.0, you can create a JUnit compatible copy of the generated Ranorex report file. The XML based JUnit test report can now easily be integrated into your CI build process.

Version Control System

Bamboo provides built-in support for source control systems like Subversion, Mercurial, Git, Perforce and CVS. For this blog we are going to use Subversion as version control of choice.

In this sample we have two solutions placed in our repository: the application we are going to test and the testing application.


To start the application under test from your test project, simply add a new ‘Run Application’ action to your recorder action table which starts the application under test using a relative path from the bin folder of the testing application.

Add RunApplication Action

Bamboo Server

Download the Bamboo installation package and follow the installation instructions.
A download link and further information about Bamboo can be found on the Atlassian homepage.

If you are going to use the default build agent running locally on the server, make sure to not start the Bamboo Server as service, but to start the server in a console as you can see in following figure. The reason is that a service does not have sufficient rights to start and access UI applications.

Start Bamboo Server From Console

After starting the server in a console, you can access the web interface of your server from the address shown in the console window (e.g. localhost:8085).

Follow the instructions of the Setup Wizard to configure your CI server.

To allow your CI process to notify the responsible developers and testers about a build success, configure your mail server (Administration -> Communication -> Mail Server).

Bamboo Configure Mail Server

Build Agent

The default installation of the Bamboo Server includes a build agent running on the local machine. You can scale out your build infrastructure on multiple machines by installing several remote build agents using following instructions: Bamboo Remote Agent Installation Guide

Note: Make sure that a valid Ranorex license as well as the Ranorex main components are installed on every Build Agent you are going to build/execute Ranorex automation code.

Add a new Plan

After setting up all necessary components, it’s time to add a new Bamboo plan.

For this purpose click the ‘+ Create Plan’ button in the upper right and choose ‘Create a New Plan’.

Bamboo Create Plan 00

Choose a project the plan will be added to and a plan name.

Bamboo Create Plan 01

Version Control Settings

Set up the path to your source repository and the build strategy.

Bamboo Create Plan 02

Configure Tasks

After automatically adding the first task which checks out the source code, add a MSBuild task for building the application under test by choosing the solution file (*.sln) of the application under test relatively to the SVN working copy root.

Bamboo Configure Tasks 01

Bamboo Add MSBuild Task-01

Bamboo Add MSBuild Task 02

Add a MSBuild task for building the testing application by choosing the solution file (*.sln) of the Ranorex Automation project relatively to the SVN working copy root.

Bamboo Configure Tasks 02

Bamboo Add MSBuild Task-01

Bamboo Add-MSBuild Task 03

Add a command task which starts the Ranorex Test Suite executable.

Bamboo Configure Tasks 03

Bamboo Add Command Task 01

Bamboo Add Command Task 02

As you can see, we defined a set of command line arguments.

In this sample the command parameter ‘/zr’, which triggers the test suite executable to generate a zipped report file and ‘/zrf:.ReportsReport.rxzlog’ which defines the name and the location of the generated zipped report file, as well as the command parameter ‘tc:/TestCase’ which specifies the test case to be executed are used. The location of the report file will later be used to define the build artifacts.

You can find a list of all available command line arguments in the section ‘Running Tests without Ranorex Studio‘ in our user guide.

The test suite executable returns ’0? on success and ‘-1? on a failure which allows the test agent to either mark this build step as successfully or failed.

After defining the tasks, the newly added plan can be enabled and created.

Bamboo Configure Tasks 04

This will trigger an initial build.

Bamboo Initial Build

Configure Artifacts

To attach a Ranorex Report to every build (= test run), you have to create an artifact definition, specifying the path and the name of the files to attach to the build.
For this purpose click ‘Configure Plan’ in the plan’s ‘Actions’ menu.

Bamboo Configure Plan

In the plan’s configuration choose the default job and the ‘Artifacts’ tab and click on the ‘Create Definition’ button.

Bamboo Create Artifact Definition

Choose a name for your definition and define the path where the Report files will be stored, relatively to the SVN working copy root (e.g. ./TestCIProject/TestCIProject/bin/Debug/Reports/). If you want to attach all files in that path to your build, you can use the copy pattern ‘*.*’. Otherwise you have to use a more specific copy pattern like ‘*.rxzlog’ which would only copy zipped report files.

Bamboo Edit Artifact Definition

Configure Notifications

To notify the responsible test engineer and/or developer about the build status, you can add email notifications. For this purpose open the ‘Notifications’ tab in the ‘Plan Configuration’.

There you can add several kinds of build notifications.

Bamboo Add Notification

Run Job

As the configuration of the plan is now finished, you can try out if everything works as expected by committing a change to either your application under test or your Ranorex project. The build will automatically be triggered and a notification will be sent to the user or email address you specified.

Bamboo Successfully Build

You can investigate the attached Report file of a specific build by opening the ‘Artifacts’ tab of the build and clicking the Report file.

Bamboo Artifact


With this step by step instruction you will be able to easily setup a Bamboo CI process executing an automated test for each committed change, thereby providing rapid feedback to the responsible test engineer and/or developer about the build success.

Please feel free to share your thoughts on this topic in the comments section.

Related Posts:

Support Corner: API Testing with Ranorex Studio and a GET Request

Support Corner: API Testing with Ranorex Studio and a GET Request

Ranorex is a powerful no-code tool that automates web, mobile, and desktop application testing. In addition to the powerful no-code recording, you can utilize the Ranorex API to create code modules in C# or VB.NET. That’s beneficial because it can increase the...

How Does Ranorex Rank on Review Sites?

How Does Ranorex Rank on Review Sites?

If you're part of a software team, you know how challenging and rewarding user interface (UI) testing can be. After all, it's a time-consuming process that involves technical knowledge and experience. But the feeling of a successful UI test—when your application runs...

Exploring 5 Types of Usability Testing

Exploring 5 Types of Usability Testing

  Usability testing gives you valuable insights into how real users interact with your product. You can use the reactions and feedback to refine your product to ensure it’s effective and enhances the user’s enjoyment. We’ll explore different types of usability...