While competing in the exacting modern software market, many companies find integrating automated testing into their overall testing process beneficial. This guide will help you better understand the different types of automated test solutions and how developers...
The common method for developing software has been shifting more and more from classical development models to more agile software development. In response to numerous inquiries from Ranorex customers, we have decided to write another Ranorex integration blog. This blog post will illustrate the integration of Ranorex Automation into a Bamboo Continuous Integration Process.
The fundamental principle of agile software development is to control changes and rapidly get response about the effects of the changes.
Applied to your Continuous Integration Process the infrastructure can be sketched as follows:
A developer or test engineer commits his changes to the version control system. Based on the committed change, the version control system triggers the continuous integration system to start a job which triggers the build agent to
- check out the modified source from the version control system,
- build the application under test (AUT),
- build the Ranorex Test Suite and
- execute the Ranorex Test Suite which tests the AUT.
The corresponding Ranorex Report file as well as a build log will be attached to the build iteration.
The developer and/or test engineer responsible for the changes will be notified about the build and test success.
Version Control System
Bamboo provides built-in support for source control systems like Subversion, Mercurial, Git, Perforce and CVS. For this blog we are going to use Subversion as version control of choice.
In this sample we have two solutions placed in our repository: the application we are going to test and the testing application.
To start the application under test from your test project, simply add a new ‘Run Application’ action to your recorder action table which starts the application under test using a relative path from the bin folder of the testing application.
Download the Bamboo installation package and follow the installation instructions.
A download link and further information about Bamboo can be found on the Atlassian homepage.
If you are going to use the default build agent running locally on the server, make sure to not start the Bamboo Server as service, but to start the server in a console as you can see in following figure. The reason is that a service does not have sufficient rights to start and access UI applications.
After starting the server in a console, you can access the web interface of your server from the address shown in the console window (e.g. localhost:8085).
Follow the instructions of the Setup Wizard to configure your CI server.
To allow your CI process to notify the responsible developers and testers about a build success, configure your mail server (Administration -> Communication -> Mail Server).
The default installation of the Bamboo Server includes a build agent running on the local machine. You can scale out your build infrastructure on multiple machines by installing several remote build agents using following instructions: Bamboo Remote Agent Installation Guide
Note: Make sure that a valid Ranorex license as well as the Ranorex main components are installed on every Build Agent you are going to build/execute Ranorex automation code.
Add a new Plan
After setting up all necessary components, it’s time to add a new Bamboo plan.
For this purpose click the ‘+ Create Plan’ button in the upper right and choose ‘Create a New Plan’.
Choose a project the plan will be added to and a plan name.
Version Control Settings
Set up the path to your source repository and the build strategy.
After automatically adding the first task which checks out the source code, add a MSBuild task for building the application under test by choosing the solution file (*.sln) of the application under test relatively to the SVN working copy root.
Add a MSBuild task for building the testing application by choosing the solution file (*.sln) of the Ranorex Automation project relatively to the SVN working copy root.
Add a command task which starts the Ranorex Test Suite executable.
As you can see, we defined a set of command line arguments.
In this sample the command parameter ‘/zr’, which triggers the test suite executable to generate a zipped report file and ‘/zrf:.ReportsReport.rxzlog’ which defines the name and the location of the generated zipped report file, as well as the command parameter ‘tc:/TestCase’ which specifies the test case to be executed are used. The location of the report file will later be used to define the build artifacts.
You can find a list of all available command line arguments in the section ‘Running Tests without Ranorex Studio‘ in our user guide.
The test suite executable returns ’0? on success and ‘-1? on a failure which allows the test agent to either mark this build step as successfully or failed.
After defining the tasks, the newly added plan can be enabled and created.
This will trigger an initial build.
To attach a Ranorex Report to every build (= test run), you have to create an artifact definition, specifying the path and the name of the files to attach to the build.
For this purpose click ‘Configure Plan’ in the plan’s ‘Actions’ menu.
In the plan’s configuration choose the default job and the ‘Artifacts’ tab and click on the ‘Create Definition’ button.
Choose a name for your definition and define the path where the Report files will be stored, relatively to the SVN working copy root (e.g. ./TestCIProject/TestCIProject/bin/Debug/Reports/). If you want to attach all files in that path to your build, you can use the copy pattern ‘*.*’. Otherwise you have to use a more specific copy pattern like ‘*.rxzlog’ which would only copy zipped report files.
To notify the responsible test engineer and/or developer about the build status, you can add email notifications. For this purpose open the ‘Notifications’ tab in the ‘Plan Configuration’.
There you can add several kinds of build notifications.
As the configuration of the plan is now finished, you can try out if everything works as expected by committing a change to either your application under test or your Ranorex project. The build will automatically be triggered and a notification will be sent to the user or email address you specified.
You can investigate the attached Report file of a specific build by opening the ‘Artifacts’ tab of the build and clicking the Report file.
With this step by step instruction you will be able to easily setup a Bamboo CI process executing an automated test for each committed change, thereby providing rapid feedback to the responsible test engineer and/or developer about the build success.
Please feel free to share your thoughts on this topic in the comments section.
Automated testing is a crucial part of software development. This guide helps you understand and implement the right types of automated testing for your needs.
Verification and validation in software testing are formal processes of assessing the correctness and completeness of a software product.
APIs and GUIs have different functions and require suitable testing. Ranorex Studio’s test automation gives you the perfect testing program for APIs and GUIs.