Modern UI testing is more than firing up an application’s website and then entering data and recording results. Today, a distributed application will be tested in a variety of release stages — from a developer’s local machine, onto a number of progressive external release environments. Such is the nature of Continuous Integration/Continuous Deployment (CI/CD).
Making it so automated UI testing can run smoothly in a CI/CD process that traverses a variety of build environments is a key requirement of the software development lifecycle in the modern enterprise.
This three-part series describes how to deploy and run GUI automation against different release environments. The first article in this series covers using Jenkins to deploy web application source code from GitHub to a build server and then perform a GUI test on the application using Ranorex’s web testing features. In this article, you’ll learn how to use Jenkins to deploy the code to a staging environment in which the web code runs in a container. We’ll use Ranorex’s web testing features to execute tests against the container. In Parts 3 and 4, we’ll deploy the containerized web application to the cloud and run Ranorex web tests against the application running under the Kubernetes cloud-native environment.
Figure 1 below shows the deployment architectures for all three stages:
Understanding the Staging Runway Architecture
The Staging runway is the deployment path for code that is checked into a project’s staging branch on GitHub. The purpose of the Staging runway is to provide a testing environment in which code that has been promoted from the development branch to the staging branch in GitHub can be subjected to functional and integration testing.
The way the Staging build process works is that once the code is promoted from the development branch to the staging branch in GitHub, the Jenkins build server takes the code from GitHub and sends it to a Windows slave server where the sample application is encapsulated into a Docker container. The containerized application runs as a web application on the Windows slave server. The web application is then tested using a Ranorex Studio web test. (See Figure 2)
The following sections show you how to create the Windows Slave server bound to a Jenkins server running on Linux. Also, you’ll learn how to create the Jenkins job that will get the code from GitHub, create the web application in a container using Docker, test it using Ranorex and then publish the test results in the Jenkins UI.
Understanding the Demonstration Application
The code we’re going to deploy and test is a trivial Node.js application that is stored on GitHub. We’re going to conduct a Ranorex Web UI Test that exercises the application’s login feature, The UI test enters user name and password data. Then it clicks the login button and verifies that the expected greeting message appears. Figure 3 below shows the demo application that’s to be tested and the Ranorex UI test that will be conducted.
Jenkins Configuration Prerequisites
Before we go into the steps required to create the Jenkins project that runs the Ranorex UI test on a Windows server, we need to make sure that the Jenkins master server has the necessary plugins installed and the Windows server has the applications needed to build and test the demonstration application.
Jenkins MSBuild Plugin
The Jenkins server requires that MSBuild Plugin is installed in order to execute the build of the Ranorex .NET test solution. The MSBuild Plugin needs to be configured with a unique label that we’ll use to identify it as the plugin to use when compiling the Ranorex test. (See Figure 4)
Jenkins Ranorex Plugin
The Jenkins server also requires that the Ranorex Plugin is installed. This plugin will run the test on the Windows server.
Jenkins Post Build Task Plugin
The Jenkins Post Build Task Plugin must be installed so that Jenkins can run cleanup tasks on the Windows Slave server after the build completes. This plugin will execute cleanup tasks regardless of whether the build succeeds or fails.
The Windows Configuration Prerequisites
The Windows Server requires that the following applications are installed directly on the host system:
Configuring the Jenkins Server with a Windows Slave
In order to run a Ranorex Studio test under the Jenkins server, we need to connect a Windows server as a slave to the Jenkins master. Once connected, Jenkins will execute the project’s build tasks on the Windows server.
Conceptually, the way the Jenkins master to Windows slave works is that we’ll configure Jenkins to be aware of the Windows server and know the location of particulars in the Windows server file system. Also, we’ll make it so the Windows slave facilitates the connection back to the Jenkins server using credential information provided by the Jenkins master server. Configuring the Windows slave back to the Jenkins master occurs on the Windows Server. Figure 5 provides a visual description of the Jenkins master to Windows Server binding interaction.
Enabling Slave Binding Using Java Web Start
From the right side of the Jenkins Server, click on Manage Jenkins to display the Manage Jenkins web page. Click Configure Global Security as shown below in Figure 6.
Creating the Windows Slave
Go back to the Manage Jenkins page and select Manage Nodes as shown in Figure 8, below.
In the textbox labeled, Remote root directory, enter the location on the Windows server where Jenkins will put its workspace folder and files. Make sure the directory exists on the Windows server.
Give the node a label by entering a value in the Labels text box. In this case, we entered the label, WinBuildServer. The label is very important. We’ll use it to tell Jenkins where to execute the tasks associated with running the Ranorex test on the Windows slave.
Select Launch agent via Java Web Start from the Launch method drop down.
If you like, you can enter a value in the text box labeled, Custom WorkDir path. Jenkins puts its runtime log files in this directory if told to do so.
Save the settings. You’ll be brought back to the Nodes page. Notice that the newly created node, WindowsSlave has a red X on the computer icon. This is because the external slave has not been bound to Jenkins master. (See Figure 11)
You’ll notice that the Jenkins web page is displaying the information needed to bind to the Jenkins master server. Jenkins is smart enough to know that the Windows slave needs to be configured to the master and show the UI accordingly. You’ll see information about how to access the agent.jar file as well as the text for the connection command for binding to the master.
Now we need to take 4 steps.
Callout 1 in Figure 12 above shows a link to agent.jar. Download the jar file to your local machine and store it in C:\Jenkins\ directory as defined previously when defining the Windows server.
Open an empty text file in your favorite editor and copy the information provided on the Jenkins web page at callout 2 into a text editor on the Windows machine. Save the text as a .bat file (callout 3) in the C:\Jenkins\directory, as shown in callout 4.
Click on the .bat file to execute the connection command. You’ll see connection binding activity as output in a command window as shown below in Figure 13.
Setting Up the Jenkins Job
We’re going to set up the Ranorex test project as a Jenkins Freestyle project. Go to the Jenkins main page and select New Item from the left side menu, as shown below in Figure 15.
Click Freestyle project, then navigate to the bottom of the web page and click the OK button. The project creation process starts. (See Figure 16)
Next, we need to tell Jenkins the location of the demo application source code on GitHub. (The code on GitHub also contains the files for the Ranorex Studio test.)
Binding to the Development Branch on Git
Navigate to the Source Code Management section of the project page. Select the Git option as shown below in Figure 18.
After we let Jenkins know where to find the source code in GitHub, we need to tell Jenkins how to be aware when new source code is available to test.
Setting Up the Build Trigger
The way we let Jenkins know that there is new code that has been committed to the development branch in GitHub is by setting a Build Trigger.
Navigate down the project configuration page to the section, Build Triggers as shown in Figure 19. Select Poll SCM and enter H/15 * * * * in the Schedule text area.
Having told Jenkins how and when to get source code from GitHub, we need to build it.
Containerizing the Node Application Using Docker
Navigate down the Jenkins project setup page to the Build section (See Figure 20). From the Add build step dropdown, select Execute Windows batch command. We need to use a Windows batch command because this job will be running on the Windows slave server. This is a very important distinction. Remember, the Jenkins server is running on a Linux machine, but the Ranorex test will execute on a Windows machine. Thus, we need to execute command line instructions that are compatible with Windows.
In this case, we’re going to tell Jenkins to encapsulate the Node.js demo app into a Docker container. The encapsulation will happen on the Windows server using a command line instruction we’ll enter into the Windows batch command text area. In the Command text area enter the text:
docker build -t feelingtracker:v1 .
Next, we’ll add another Windows batch command task in Jenkins into which we’ll enter the code to create and run the container that contains the demo app.
In the text area of the newly added task enter the command:
docker run -d --name myapp_container -p 3000:3000 -e DEPLOYMENT_ENV=STAGING -e CURRENT_VERSION=v1 feelingtracker:v1
See Figure 21 below.
Once we have the demonstration application up and running in a container, we need to test it. First, we need to build the Ranorex Studio test suite.
Building the Ranorex Studio Test Suite
We’re going to build the Ranorex test suite under a new build step. You’ll see an Add build step drop-down button beneath the previous step we built. Click that button and select, Build a Visual Studio project or solution using MSBuild, as shown below in Figure 22.
In the MSBuild Build File text box, enter the path to the Ranorex solution file (.sln) that will drive the build compilation. In this case, the entire Ranorex build solution is stored in the GitHub project, in the directory \ranorex\feelingTrackerRanorexTest. (See Figure 23)
Now we need to start the demonstration application.
Running the Ranorex Studio test case
Add a new build step to the Jenkins project. But this time, select Run a Ranorex test suite, from the build steps drop-down menu, as shown below in Figure 24. (This assumes you have installed the Ranorex plugin previously, as required.)
Enter a report name in the textbox, Ranorex report file name. Also, click the checkbox for JUnit-compatible report, as shown above in Figure 24.
This step will execute the Ranorex test on the Windows server and generate an XML file, BasicReport.rxlog.junit.xml which describe the results of the Ranorex test. We’ll use this file in a later step.
Cleaning Up After the Build
After the test has run, we need to remove from the Windows Server both the running container and the Docker image that was used to build the container. We’re going to accomplish these tasks by using a Jenkins post-build task.
Navigate to the end of the Jenkins job we’ve built and click the button Add post-build action as shown below in Figure 24 (callout 1). Then select, Post build task from the drop-down list (callout 2).
The text, \. is a Java regular expression that indicates any character (See Figure 26, callout 1). Jenkins will apply the regular expression to the log file text that is generated during the build. Operationally we’re telling Jenkins to go into the log file and identify any character at all. Once any character is identified, execute the Post-build task. Effectively we’re having Jenkins execute the Post-build task under any condition, regardless of whether the build was successful or failed. However, there might be other situations later on in which you’ll want to execute a Post-build task under a certain condition, for example, a FAILURE. In a case such as this, you’ll create regular expression entry in Log text textbox that identifies log output that indicates the type of failure you’re interested in.
For this task we want the container to stop running under any condition, pass or fail. Thus, after entering the “any character” regular expression, we enter the text docker stop myapp_container in the Script textbox as shown below in Figure 26, callout 2.
Add another Post build task to the build and enter the regular expression text, \. into the Log text textbox. (See Figure 28, callout 1, below). Remember, we need to enter the regular expression for “any character” in order to have the Post-build task execute regardless of success or failure.
Enter the Docker command, docker image rm feelingtracker:v1 in the Script textbox as shown below in Figure 28, callout 2.)
Next, we need to add another post-build action that reports the results of the Ranorex test.
Defining Reports as Post Build Task
Click the Post-build Actions button and select, Publish JUnit test result report. (See Figure 29)
Setting up the build is complete. Click the Save button at the bottom of this build configuration page. Now it’s time to run the build.
To run the build, click the link Build Now on the left side menu of the Web page of the build project page, as shown below in Figure 30.
In this article, we looked at how to run a Ranorex test in the Staging runway of the release process. We ran the tests on a Windows slave bound to the master Jenkins server.
The next step in the multi-phase GUI testing process is to go to the UAT release phase. In this phase, we’ll create a new Jenkins build project that gets the demonstration application code from the Master branch of the GitHub repository. The Jenkins build process for the UAT phase will send the containerized demonstration application to a Kubernetes cluster running on Google Cloud and then use Ranorex to test the demonstration application’s UI as it’s running in the Kubernetes Cluster.