In software engineering, continuous integration (CI) means the continuous application of quality control processes — small units of effort, applied frequently.
In this blog, we’ll show you how to set up a CI job with Hudson/Jenkins that automatically builds and executes Ranorex automated tests and generates test reports for every change committed to a Subversion repository.
Mastering Jenkins Integration: See how easy it is to integrate Ranorex automated tests into your Jenkins build management and deployment process.
Continuous integration has many advantages. When tests fail or bugs emerge, developers can revert the codebase to a bug-free state without wasting time debugging. Developers can detect and fix integration problems continuously—and thus avoid last-minute chaos at release dates. In addition, CI testing provides:
Below are download links and installation description for Hudson and Jenkins:
This blog post will use Jenkins as the CI tool. Differences between Jenkins and Hudson are minimal. Note that because Jenkins or the nodes executing the CI jobs normally are started as Windows services, they do not have sufficient rights to start UI-applications. Ensure that Jenkins is not started as a service on the master or on the slave nodes where the Ranorex automation should be triggered.
For the Jenkins master, open the Services tool (which is part of the “Administrative Tools” in the control panel), choose the Jenkins service, stop the service, and set the Startup type to Disabled, as shown below.
Use the following command to start Jenkins manually from the installation folder:
java -jar jenkins.war
After starting Jenkins, use this address to access the web interface: http://localhost:8080/
To configure your Jenkins server, navigate to the Jenkins menu and select Manage Jenkins > Configure System:
Note: It is necessary to have the Ranorex main components—and a valid Ranorex license—installed on each machine where you want to build and execute Ranorex code.
As mentioned before, this example uses a Subversion repository as the base for the continuous integration process. In this sample, we have two solutions in the repository: the application under test and the automated Ranorex tests.
To start the application under test from your test project, simply add a new Run Application action to your action table in Ranorex Studio, which starts the application under test, using a relative path to the repository root:
Since we want to build the code for each committed change within the SVN repository, we need a Subversion and an MS Build plugin for Jenkins. An additional mail plugin will ensure that an email is sent with each build.
From the Jenkins menu, choose Manage Jenkins > Manage Plugins. Select the following plugins from the list of available plugins and install them if they are not already installed:
The installed plugins also need to be configured, following the steps below:
Now that the system is configured, you can add a new Jenkins job. This job will update the checked-out files from an SVN repository, build both the application under test and the Ranorex automation project, execute the application under test as well as the automation code and send an email with the report file attached.
Start by creating a new item. Choose Build free-style software project as
The next step is to check out the source of both the application under test and the test automation project. Start with choosing Subversion as the source code management tool. Then, enter the repository containing your application under test as well as your test automation project. Finally, select Use ‘svn update’ as much as possible as the check-out strategy:
With this configuration, the application under test as well as the test automation project will be checked out and updated locally.
Now that the source code management is configured, you can start with processing the updated files. First, add MSBuild steps for both projects:
Choose the configured MSBuild version and enter the path of the solution file relative to the repository root (the workspace folder of the Jenkins job), for both the automated and the automating project:
By adding these two build steps, the executables will be built automatically. Now the newly-built application should be tested. This can be accomplished by adding a new “Run a Ranorex test suite” build step that starts the test suite executable.
To see how this works, watch the following short video or follow the steps below:
How to configure the “Run a Ranorex test suite” build step
The test suite executable returns “0” on success and “-1” on failure. Based on this return value, Jenkins will mark the build as successful or failed.
If you selected the “JUnit-compatible report” option in the previous step, now you can publish a JUnit test result report to the Jenkins build after the automated tests complete. In this step, you will add a “Publish Junit test result report” post-build action to the build job and define the test report
In addition, you can send an email regarding the success of the triggered build. This email should include the zipped report file, mentioned before, as an attachment. To do so, add the new post-build action “Editable Email Notification”, choose the report file location defined before as attachment, and add triggers for each job status you want to be informed about. In this sample, an email will be sent if a job has failed or succeeded.
Once you have completed these steps and saved your changes, verify that everything works as expected by clicking Build now:
After running the generated job, you should see all finished builds within the build history. Icons indicate the status of the individual builds. If you click on a specific build, you will see the detailed JUnit test results. View the zipped report files of all builds by opening them in the local workspace (“Workspace/Reports”):
As configured in the previous steps, an email will be sent to the specified email address(es), including the console output in the email text as well as the generated zipped report file as an attachment.
Now we can manually trigger a build. Because we are working with Subversion in this example, it would be beneficial to trigger the script for each commit. To do so, add a server-side repository hook, which automatically triggers Jenkins to start a new build for each change committed, as described in the Subversion plugin documentation.
Alternatively, you can activate polling of the source code management system as a build trigger in your Jenkins job configuration. As shown in the following screenshot, you can define the interval, after which the source code management will be invoked (e.g. 5 minutes after every full hour):
Following the steps above, you will be able to set up a continuous integration process that includes Ranorex automated tests for the application under test. Each commit will trigger an automated test run. Once the test run has finished, you’ll instantly receive an email with the Ranorex test report.
Note: This blog was originally published in July 2012 and has been revised to reflect recent technical developments.
Hosted and on-prem solutions for Git, SVN and Perforce
Cross-Technology | Cross-Device | Cross-Platform
Ranorex Studio is renowned for its robust no-code capabilities, which allow tests to be automated seamlessly across web, mobile, and desktop applications. Beyond its intuitive recording features, Ranorex Studio allows custom code module creation using C# or VB.NET,...
It’s difficult for any IT organization to execute DevOps effectively without test automation. However, it’s often easier said than done. Overcoming the challenges of automated software testing can end up slowing down product delivery and impacting quality, the exact...
There are more and more Android testing tools available for mobile app developers. These are our favorites for performance, accessibility, and security.
