Integrate Automated Testing Into Any Continuous Integration Process

Oct 20, 2021 | Integration

continuous integration test automation

Long gone is the time of waterfall’s strictly separated development & testing phases. Today, it’s all about fast feedback, quick iterations and frequent releases at a previously unseen velocity. It requires an agile methodology to keep up with the high demands. Your team’s success depends on a supporting infrastructure with the right tooling. Without any doubt, automation plays an essential role here. Our tip: Integrate test automation into your continuous integration (CI) process.

We wouldn’t want you to waste precious time if your development environment is already set up. That’s why you can integrate Ranorex into any continuous integration process. Let’s have a closer look at the benefits of integrating test automation into your CI system, and how you can do it.

Integration-Overview

Automated testing and continuous integration

The idea of continuous integration is to frequently promote code changes and rapidly get feedback about the impact these changes have on the application or system. Including test automation in the development cycle enables you to automatically test each incremental code change.

So basically every time a developer commits code changes to the version control system (VCS) such as Git or TFVC, a build of the application under test as well as the Ranorex test automation project is triggered in the CI system. The resulting test automation executable then runs against the application under test.

To evaluate the outcome of the automated test, the continuous integration tool examines the return value of the executable or its output text (e.g. “TEST FAILED” for failure). With Ranorex, the return value ‘0′ signals the successful execution of the GUI test script, while the return value ‘-1′ signals a failure. Each team member automatically receives a notification about a finished build. This notification includes build logs as well as a test execution report.

Advantages of integrating Ranorex into your CI system:

  • As each code change is immediately tested, potentially introduced bugs are found faster, which ultimately makes it easier to fix them.
  • The test automation report enhances transparency, as each team member will receive instant feedback about the state of code.
  • There’re no integration problems, as Ranorex can be integrated into any CI tool.

Setting up Ranorex test automation in your CI system

Note: You must install Ranorex on each machine you’d like to execute Ranorex tests on. You’ll need a valid license to do so. Please find more information about Ranorex licenses on our dedicated Pricing page.

Each committed change in the application under test and the test automation project should be automatically tested. In other words, every change should trigger these 3 steps:

  • building the application under test
  • building the Ranorex test suite
  • executing the Ranorex test suite
Integration-CI-in-Detail11

First, you need to manually set up these steps in your CI system.

1. Build the application under test

The first build step should generate an executable of your application under test. This executable should later be triggered from the Ranorex test suite project. Therefore, add a build step which will build your application under test (e.g. MSBuild build step, Ant build step, …).

2. Build the Ranorex test suite

In this second step, you’ll need to generate an executable to automate your application under test. To do so, add a build step (MSBuild or Visual Studio) and choose the project file (*.csproj) of your Ranorex project which should be built.

3. Execute the Ranorex test suite

The third step should execute the previously created executables. Simply add an execution step triggering the *.exe file of the test automation project and define the command line arguments if needed.

The test execution should now be triggered on the same system the projects were built on. If you want to trigger the execution on another system, you need to deploy the built executables and all connected files to that system. Please make sure to execute the application under test and the Ranorex test suite in a desktop and not in a console session.

Automated testing of frequent code changes

If the code of your application under test or your test automation project changes frequently, it doesn’t make sense to run the entire test suite including all test cases with every build. Instead, you should run only those test cases that are affected by the changes. How? Run configurations!

You can add and edit run configurations directly in the test suite, as described in our User Guide.

You can trigger a run configuration using a command line argument. The following command line, for example, will run the test suite executable ‘TestCIProject’ with the run configuration (/rc) ‘SmokeTest’ and generate a zipped report file (/zr /zrf) ‘Report.rxzlog’ in the folder ‘/Reports/’.

TestCIProject.exe /rc:SmokeTest /zr /zrf:Reports/Report.rxzlog

For more information on command line arguments in Ranorex Studio, refer to our User Guide.

Test automation report – the importance of feedback

“No news is good news” is definitely not true for agile teams. It’s important that everyone in a team – whether it is a developer or tester – knows about the state of the code and, thus, the outcome of the automated test run. It really couldn’t be any easier: Simply add a post build action which sends an email to your team members with the build log and the generated zipped report attached.

Note: In Ranorex Studio versions 7.0 and later, you can create a JUnit-compatible copy of the generated Ranorex report file. The XML based JUnit test report can now easily be integrated into your CI build process.

Integrate Ranorex into a specific CI system:

You’re using a specific CI tool? Whether it’s Bamboo, Jenkins, HP Quality Center, TeamCity or Microsoft Test Manager – check out the section below for detailed instructions on how to integrate Ranorex into your CI tool!

As you can see, it’s easy to incorporate Ranorex test automation in your continuous integration process. Each code change in your application under test and your test automation project will be automatically tested, which enhances transparency and enables you to find bugs faster.

Ready to experience the benefits of test automation in your continuous integration process? Download the full-featured Ranorex Studio trial today and try it out in your own environment.  Our pre-sales team and getting started resources are available to help ensure you have a successful experience. Have fun integrating!

All-in-one Test Automation

Cross-Technology | Cross-Device | Cross-Platform

Related Posts:

Effective Black Box Testing Methods You Need to Try

Effective Black Box Testing Methods You Need to Try

When users open software solutions, they expect them to function as needed. For example, when a business analyst opens Excel, they hope to work with data without requiring knowledge of what’s happening with the application internally. If something breaks, they won’t...

8 Steps to Create a Data Migration Plan

8 Steps to Create a Data Migration Plan

When companies change systems or move information to a more secure location, they typically need to perform a data migration. If a company wants to use cloud-based solutions, it must transfer existing information from localized hardware to a cloud environment. A...