Long gone is the time of waterfall’s strictly separated development & testing phases. Today, it’s all about fast feedback, quick iterations and frequent releases at a previously unseen velocity. It requires an agile methodology to keep up with the high demands. Your team’s success depends on a supporting infrastructure with the right tooling. Without any doubt, automation plays an essential role here. Our tip: Integrate test automation into your continuous integration (CI) process.
We wouldn’t want you to waste precious time if you’ve your development environment already set up. That’s why you can integrate Ranorex into any continuous integration process. Let’s have a closer look at the benefits of integrating test automation into your CI system, and how you can do it:
Automated testing and continuous integration
The idea of continuous integration is to frequently promote code changes and rapidly get feedback about the impact these changes have on the application or system. Including test automation in the development cycle enables you to automatically test each incremental code change.
So basically every time a developer commits code changes to the version control system (VCS) such as Git or TFVC, a build of the application under test as well as the Ranorex test automation project is triggered in the CI system. The resulting test automation executable then runs against the application under test.
To evaluate the outcome of the automated test, the continuous integration tool examines the return value of the executable or its output text (e.g. “TEST FAILED” for failure). With Ranorex, the return value ‘0′ signals the successful execution of the test script, while the return value ‘-1′ signals a failure. Each team member automatically receives a notification about a finished build. This notification includes build logs as well as a test execution report.
Advantages of integrating Ranorex into your CI system:
- As each code change is immediately tested, potentially introduced bugs are found faster, which ultimately makes it easier to fix them.
- The test automation report enhances transparency, as each team member will receive instant feedback about the state of code.
- There’re no integration problems, as Ranorex can be integrated into any CI tool.
Setting up Ranorex test automation in your CI system
Note: You have to install Ranorex on each machine you’d like to execute Ranorex tests on. You’ll need a valid license to do so. Please find more information about Ranorex licenses on our dedicated Pricing page.
Each committed change in the application under test and the test automation project should be automatically tested. In other words, every change should trigger these 3 steps:
- building the application under test
- building the Ranorex test suite
- executing the Ranorex test suite
First, you need to manually set up these steps in your CI system.
1. Build the application under test
The first build step should generate an executable of your application under test. This executable should later be triggered from the Ranorex test suite project.
Thus, add a build step which will build your application under test (e.g. MSBuild build step, Ant build step, …).
2. Build the Ranorex test suite
In this second step, you’ll need to generate an executable to automate your application under test. To do so, add a build step (MSBuild or Visual Studio) and choose the project file (*.csproj) of your Ranorex project which should be built.
3. Execute the Ranorex test suite
The third step should execute the previously created executables. Simply add an execution step triggering the *.exe file of the test automation project and define the command line arguments if needed.
The test execution should now be triggered on the same system the projects were built on. If you want to trigger the execution on another system, you need to deploy the built executables and all connected files to that system. Please make sure to execute the application under test and the Ranorex test suite in a desktop and not in a console session.
Automated testing of frequent code changes
If the code of your application under test or your test automation project changes frequently, it doesn’t make sense to run the entire test suite including all test cases with every build. Instead, you should run only those test cases that are affected by the changes. How? Run configurations!
You can add and edit run configurations directly in the test suite (see user guide section ‘Running a Test Suite’).
You can trigger a run configuration using a command line argument. The following command line, for example, will run the test suite executable ‘TestCIProject’ with the run configuration (/rc) ‘SmokeTest’ and generate a zipped report file (/zr /zrf) ‘Report.rxzlog’ in the folder ‘/Reports/’.
TestCIProject.exe /rc:SmokeTest /zr /zrf:Reports/Report.rxzlog
Interested in more command line arguments? You find more in the user guide section ‘Running Tests without Ranorex Studio‘.
Test automation report – the importance of feedback
“No news is good news” is definitely not true for agile teams. It’s important that everyone in a team – whether it is a developer or tester – knows about the state of the code and, thus, the outcome of the automated test run. It really couldn’t be any easier: Simply add a post build action which sends a mail to your team members with the build log and the generated zipped report attached.
Integrate Ranorex into a specific CI system:
You’re using a specific CI tool? Whether it’s Bamboo, Jenkins, HP Quality Center, TeamCity or Microsoft Test Manager – check out the section below to find a detailed instruction on how to integrate Ranorex into your CI tool!
- Integrate Ranorex with Bamboo
- Integrate Ranorex with TeamCity
- Integrate Ranorex with Jenkins
- Integrate Ranorex with HP Quality Center
- Integrate Ranorex with Microsoft Test Manager
As you can see, it’s easy to integrate Ranorex test automation in your continuous integration system. Each code change in your application under test and your test automation project will be automatically tested, which enhances transparency and enables you to find bugs faster.
You want to know about the benefits of integrating Ranorex into your development environment? Try it out! Download the full-featured 30-day Ranorex trial and see the benefits for yourself! Have fun integrating!