Making it so automated UI testing can run smoothly in a CI/CD process that traverses a variety of build environments, is a key requirement of the software development lifecycle in the modern enterprise.
This four-part series describes how to deploy and run GUI automation against different release environments. This first part will cover using Jenkins to deploy web application source code from GitHub to a build server and then perform a GUI test on the application using Ranorex’s web testing features.
The second part will show how to use Jenkins to deploy the code to a staging environment in which the web code runs in a container. We’ll use Ranorex’s web testing features to execute tests against the container.
Finally, in Parts 3 and 4, we’ll deploy the containerized web application to the cloud and run Ranorex web tests against the application running under Kubernetes cloud-native environment.
Figure 1 below shows the deployment architectures for all three stages.
The Development runway is the deployment path for code that is checked into a project’s development branch on GitHub. The purpose of the Development runway is to provide an application environment in which code from all the developers is unified into a single code base that can be reviewed and tested.
The way the Development build process works is that once the developer checks code into the GitHub repository’s development branch from his her local machine, the Jenkins build server takes the code from GitHub and sends it to a Windows slave server where the sample application is built, started and tested using a Ranorex web test. (See Figure 2.)
The following sections show you how to create the Windows Slave server bound to a Jenkins server running on Linux. Also, you’ll learn how to create the Jenkins job that will get the code from GitHub, test it using Ranorex and then publish the test results in the Jenkins UI.
The code we are going to deploy and test is a trivial Node.js application that is stored on GitHub. We’re going to conduct a Ranorex Web UI Test that exercises the application’s login feature, The UI test enters username and password data. Then it clicks the login button and verifies that the expected greeting message appears. Figure 3 below shows the demo application that’s to be tested and the Ranorex UI test that will be conducted.
Before we go into the steps required to create the Jenkins project to run Ranorex UI test on a Windows server, we need to make sure that the Jenkins master server has the necessary plugins installed and the Windows server has the applications needed.
The Jenkins server requires that MSBuild Plugin is installed in order to execute the build of the Ranorex .NET test solution.
The Jenkins server also requires that the Ranorex Plugin is installed. This plugin will run the test on the Windows server.
The Windows Server requires that the following applications are installed directly on the host system:
In order to run a Ranorex test under the Jenkins server, we need to connect a Windows server as a slave to the Jenkins master. Once connected, Jenkins will execute the project’s build tasks on the Windows server.
Conceptually, the way the Jenkins master to Windows slave works is that we’ll configure Jenkins to be aware of the Windows server and know the location of particulars in the Windows server file system. Also, we’ll make it so the Windows slave facilitates the connection back to the Jenkins server using credential information provided by the Jenkins master server. Configuring the Windows slave back to the Jenkins master occurs on the Windows Server. Figure 4 provides a visual description of the Jenkins master to Windows Server binding interaction.
From the right side of the Jenkins Server, click on Manage Jenkins to display the Manage Jenkins web page. Click Configure Global Security as shown below in Figure 5.
Go back to the Manage Jenkins page and select Manage Nodes as shown in Figure 7, below.
In the textbox labeled Remote root directory, enter the location on the Windows server where Jenkins will put its workspace folder and files. Make sure the directory exists on the Windows server.
Give the node a label by entering a value in the Labels textbox. In this case, we entered the label, WinBuildServer. The label is very important. We’ll use it to tell Jenkins where to execute the tasks associated with running the Ranorex test on the Windows slave.
Select Launch agent via Java Web Start from the Launch method drop-down.
If you like, you can enter a value in the text box labeled Custom WorkDir path. Jenkins puts its runtime log files in this directory if told to do so.
Save the settings. You’ll be brought back to the Nodes page. Notice that the newly created node, WindowsSlave has a red X on the computer icon. This is because the external slave has not been bound to Jenkins master. (See Figure 10.)
You’ll notice that the Jenkins web page is displaying the information needed to bind to the Jenkins master server. Jenkins is smart enough to know that the Windows slave needs to be configured to the master and show the UI accordingly. You’ll see information about how to access the agent.jar file as well as the text for the connection command for binding to the master.
Now we need to take 4 steps.
Open an empty text file in your favorite editor and copy the information provided on the Jenkins web page at Callout 2 into a text editor on the Windows machine. Save the text as a .bat file (Callout 3) in the C:\Jenkins\ directory, as shown in Callout 4.
Click on the .bat file to execute the connection command. You’ll see connection binding activity as output in a command window as shown below in Figure 12.
We’re going to setup the Ranorex test project as a Jenkins Freestyle project. Go to the Jenkins main page and select New Item from the left side menu, as shown below in Figure 14.
Click on Freestyle project, then navigate to the bottom of the web page and click the OK button. The project creation process starts. (See Figure 15)
Navigate to the bottom of the General tab to the checkbox Restrict, where this project can be run.
Next, we need to tell Jenkins the location of the demo application source code on GitHub. (The code on GitHub also contains the files for the Ranorex test.)
Navigate to the Source Code Management section of the project page. Select the Git option as shown below in Figure 17.
After we let Jenkins know where to find the source code in GitHub, we need to tell Jenkins how to be aware when new source code is available to test.
The way we let Jenkins know that there is new code has been committed to the development branch in GitHub is by setting a Build Trigger.
Navigate down the project configuration page to the section, Build Triggers as shown in Figure 18. Select Poll SCM and enter H/15 * * * * in the Schedule text area.
Having told Jenkins how and when to get the source code from GitHub, we now need to build it.
Navigate down the Jenkins project setup page to the Build section. (See Figure 19.) From the Add build step dropdown, select Execute Windows batch command. We need to use a Windows batch command because this job will be running on the Windows slave server. This is a very important distinction. Remember, the Jenkins server is running on a Linux machine, but the Ranorex test will execute on a Windows machine. Thus, we need to execute command line instructions that are compatible with Windows.
In this case, we’re going to tell Jenkins to build the Node.js demo app on the Windows server using the command line instruction. In the Command text area enter the text npm install.
We’re going to build the Ranorex test suite under a new build step. You’ll see an Add build step drop-down button beneath the previous step we built. Click that button and select, Build a Visual Studio project or solution using MSBuild, as shown below in Figure 20.
In the MSBuild Build File text box, enter the path to the Ranorex solution file (.sln) that will drive the build compilation. In this case, the entire Ranorex build solution is stored in the GitHub project, in the directory \ranorex\feelingTrackerRanorexTest. (See Figure 21)
Now we need to start the demonstration application.
Add another Windows batch command to the Jenkins build process, as we did previously in Figure 19. This command will start the demonstration app. In the Command textarea enter: forever start app.js (See Figure 22. Remember, having the forever Node package installed on the Windows server is a configuration prerequisite for running this Jenkins project.)
Now that the demo application is up and running we can conduct the Ranorex test.
Add a new build step to the Jenkins project. But this time, select Run a Ranorex test suite, from the build steps drop-down menu, as shown below in Figure 23. (This assumes you have installed the Ranorex plugin previously, as required.)
Enter a report name in the textbox, Ranorex report file name. Also, click the checkbox for JUnit-compatible report, as shown above in Figure 23.
This step will execute the Ranorex test on the Windows server and generate an XML file, BasicReport.rxlog.junit.xml, which describes the results of the Ranorex test. We’ll use this file in a later step.
After the test has run, we need to shut down all instances of Node.js and Chrome running on the Windows server. We need to do this so that subsequent test runs on the Windows server are not subject to preexisting conditions that create errors. For example, trying to start the demo application’s web server will error, if the application is already running.
To shut down all instances of Node.js running on the Windows server, add a new Windows command build step, and in the Command text area, enter taskkill /im node.exe /F, as shown in Figure 24 below.
You’ll see the Post-build Actions section at the bottom of the Jenkins project page. Click the Add post-build action dropdown button and select Publish JUnit test result report. (See Figure 26)
Setting up the build is complete. Click the Save button at the bottom of this build configuration page. Now it’s time to run the build.
To run the build, click the link Build Now on the left side menu of the Web page of the build project page, as shown below in Figure 27.
In this article, we looked at how to run a Ranorex test in the Development runway of the release process. We ran the tests on Windows slave bound to the master Jenkins server.
The next step in the multi-phase GUI testing process is to go to the Staging phase. In this phase, we’ll create a new Jenkins build project that gets the demonstration application code from the Staging branch of the GitHub repository. The Jenkins build process for this stage will be to run the demo application from within a Docker container and make it so the Ranorex UI tests execute against the running container instance.
There are more and more Android testing tools available for mobile app developers. These are our favorites for performance, accessibility, and security.
Regression testing and retesting are essential methodologies testers employ to ensure software quality. If you’re new to both or aren’t sure when to use which technique, this article should help. We’ll discuss the importance of regression testing in software testing. ...
DevOps test automation enables teams to work faster and create better products. These best practices and tools make it easier to implement test automation.
