Multi-Phase GUI Testing with Ranorex under Cloud-Based CI/CD Part 4: Testing

This article is the last entry in a 4-part series that describes how to use Jenkins and Ranorex Studio to implement continuous deployment of a Node.js application from a source code repository on GitHub into a Kubernetes cluster running on Google Cloud. (See Figure 1)

The prior article describes how to package up the Node.js application, FeelingTracker, into a Docker container and then deploy it to the Kubernetes cluster. In this article, you’ll learn how to determine the IP address of the demonstration Node.js application running as a Kubernetes service on Google Cloud and then run the predefined Ranorex test against a URL that uses the discovered IP address.

The process to test the Node.js application is spread over two Jenkins projects. The first project encapsulates the Node.js application into a Kubernetes container image and then creates a container from that image on Google Cloud. This project was presented in the previous article in this series.

Once the deployment is complete, a post-build event in Jenkins calls a second Jenkins project. This second project — the subject of this article — contains the information and logic necessary to test the containerized application running on Google Cloud. Figure 2 below shows the Jenkins post-build event from the first project that calls the second build project which has the name, RanorexFeelingTrackerTest.

Configuring the Ranorex Project to Accept a Variable from the Command Line

To be able to test the Node.js application running on Google Cloud, we need to modify the Ranorex test we’ve been using. In the testing scenarios described in previous articles, the location of the Node.js application under test was localhost, on the same machine where the Ranorex test was run. Hence, we could hard code the location URL of the Node.js in the prior Ranorex test to be simply, localhost. However, now that we are deploying the Node.js application to a Kubernetes cluster on Google Cloud, the IP address is known only at the time of deployment. Thus, we need to configure the Ranorex test to accept the discovered IP address as a command line argument that gets that passed to the Ranorex test when executed from within Jenkins.

Binding the Ranorex test to accept a command line argument is a two-step process. First, we need to create a module level variable that is referenced within each test action. An example of a test action is to have Ranorex open the browser at the location of the Node.js application on Google Cloud.

Then, once a module variable is defined within the test actions, the module variable needs to be bound to a global variable defined for the Ranorex test project. Jenkins will use the global variable’s name as the command line argument passed to Ranorex when the test is called within the Jenkins build project.

Figure 3 below shows the process of defining the module variable, $testUrl for the open action in the Ranorex test associated with the article (you can view the code to the entire test on GitHub). The process is similar for all the other actions in the test that need to know the location of the Node.js application on Google Cloud.

Figure 4 shows the results when the module variable $testUrl is assigned to actions in the module, OpenBrowser.rxrec.

Once the module level variable is set, it needs to be bound to a global variable. We define a global variable from within the test’s page in Ranorex Studio. To display the test page, go to the solution list as shown above in Figure 3 and double click the test project, in this case, feelingTrackerRanorexTest.

This will open the test’s page in Ranorex Studio. Right-click on the test as shown below in Figure 5 and select Global parameters.

Select Add row… from the Global parameters dialog that appears, as shown in Figure 6 below. Add the global parameter, webAppUrl, and leave it unbound.

The global parameter webAppUrl is the command line argument that will be set by Jenkins. Now we need to bind the global parameter, webAppUrl to the module variable, testUrl, as follows:

1. Go back to the test page in Ranorex Studio as shown in Figure 7 below.
2. Right-click the test case and select Data binding… in the context menu that appears. The TestCase properties dialog appears.
3. Enter the exact declaration of the global parameter in the name column, according to test suite as shown in Figure 7. In this case, the declaration is feelingTrackerRanorexTest.webAppUrl.
4. Then, to apply to the global parameter to the module variables, check off the usages of the module variable as they appear in the module variable column.

Remember that you can view this configuration in the Ranorex UI by opening the demonstration test for the app in GitHub.

Exposing the global parameter provides a way for Jenkins to communicate to the Ranorex the location on the internet of the Node.js application under test when the IP address is discovered. Once we’ve finished modifying the Ranorex test, we’re ready to create the Jenkins build project.

Windows Build Server Configuration

As mentioned previously, we’re going to test the Node.js application using a separate Jenkins build from the one that deployed the application. Thus, we create a new project according to the information provided in Figure 8.

Given that Ranorex needs to run in a Windows environment, we’ll configure Jenkins to run the build tasks on the Windows slave server we’ve configured previously under Jenkins. We declare that Jenkins use the slave server by checking the checkbox, Restrict where the project can be run, as shown above in Figure 8.

The test will need access credential information in order to be able to make the calls to Kubernetes necessary to determine IP address of the Kubernetes service under which the Node.js application is running. This access information is stored in a set of Jenkins secrets we configured previously in Jenkins server. But still, we need to tell this project to use those secrets. Thus, we check the checkbox, Use secret text(s) or files(s) as shown below in Figure 9.

This project needs to access source code in order to get the Ranorex test to compile and run. Therefore, we set the Source Code Management pane in Jenkins as shown in Figure 10, below.

Jenkins uses the Binding plugin to bind secrets to local variables. The mechanics of the process are straightforward. You declare a variable then bind it to a specific secret that was created earlier. Figure 11 below shows the declaration of three variables, GC_ACCESS, GC_SERVICE_ACCOUNT, and PROJECT_ID. These variables are bound to the secrets, GC_FEELINGTRACKER_ACCESS, GOOGLE_SERVICE_ACCOUNT and FEELING_TRACER_PROJECT_ID, respectively.

These variables will be used in the steps that follow. As a matter of good housekeeping, we need to set the timeout period for the test to make sure it does not run on forever. Setting the timeout period is done with the Time-out strategy section of the Jenkins UI as shown below in Figure 12.

We bind the Ranorex test Visual Studio solution to the Jenkins build from within the Build section of the Jenkins UI. Given the configuration provided in Figure 13, below, Jenkins will use MSBuild to compile the solution, feelingTrackerRanaorexText.sln into the binary executable that represents the Ranorex test that will be run against the Node.js application running on Google Cloud.

Our next series of steps involves executing the Windows batch commands necessary to run the Ranorex test against the Node.js application running in the Kubernetes cluster on Google Cloud.

Binding Jenkins to the Kubernetes Instance on GC

In order to access Google Cloud to get the information we need about the Kubernetes cluster, we need to make the access credentials available to Jenkins. One way to get access to Google Cloud is to use a technique defined by Google in which access information is made available to the local environment by way of a JSON file that’s generated by Google Cloud. This JSON file is stored in the location on the Windows slave server from which Google SDK command is being executed.

In order to follow proper security practices, we copied the contents of the Google-generated JSON file into the Jenkins secret GC_FEELINGTRACKER_ACCESS which is associated with the Jenkins variable GC_ACCESS. Then when the Jenkins job is run, the secret access information is injected into a file locally on the Windows slave server. We use the Linux echo command to copy the contents of GC_ACCESS (the secret access information) to the local file, client-secret.json. See Figure 14, below.

We allow Jenkins to have access to Google Cloud by running the command, gcloud auth activate-service-account –key-file client-secret.json as shown in Figure 15 below.

Figure 16 shows a command that will display the value of the environment variable PROJECT_ID as output as the project builds. Also, the information will be added to builds log file.

Displaying the PROJECT_ID is done as a matter of convenience. The value of PROJECT_ID is not particularly sensitive in terms of security. However, using echo to output the value of environment variables at run time is a technique to be used cautiously. You should never output any information that allows system access. Once information is written to log, it can be discovered by bad actors if security precautions are lax.

Figure 17 below shows the Google SDK command, gcloud being used in a Jenkins Windows batch command. This command, gcloud container clusters get-credentials, downloads the Kubernetes config file that Jenkins will use in conjunction with the Kubernetes command kubectl, to access the pre-existing Kubernetes cluster, feelingtrackercluster out on Google Cloud. This cluster was created in the previous Jenkins build project that deployed the Node.js application to Google Cloud.

Once the Jenkins build has access to Kubernetes clusters on Google Cloud, we need to tell Jenkins which Kubernetes cluster to use. Accessing the cluster to use is done by declaring the particular Kubernetes configuration that is associated with the cluster of interest. In terms of the Google Cloud SDK, clusters are organized according to the Google Cloud Project ID. Figure 18 below shows how to use the SDK command gcloud config set to bind the Jenkins build project to the Kubernetes cluster that has the Kubernetes service that represents the Node.js application.

We’ve given Jenkins access to the Kubernetes cluster that is running the Node.js application. Now we need to determine the IP address of the Kubernetes service under which the application is running.

Finding The FeelingTracker Kubernetes Service on Google Cloud

To get the IP address of the Kubernetes service under which the application is running, we’re going to query Kubernetes directly. Before we do this, let’s take a moment to review how Kubernetes represents an application via service.

In Kubernetes, the actual running of an application takes place in one or many Docker containers that are encapsulated into a Kubernetes object called a pod. Another Kubernetes object, called a service, routes input from the outside world to the given pod. Also, the output from the pod to the outside world is sent via the service.

When a service is created, Kubernetes will assign an IP address to the service that is accessible to the outside world. It takes time for the IP address assignment to happen after the service has been created. This timespan can sometimes take minutes. Thus, we need to make Jenkins wait a bit before it goes out to Google Cloud to get the service IP information from Kubernetes.

Figure 19 below show a simple technique in Windows by which the ping command is used to make the Jenkins job wait. In the case of Figure 18, we’re having Jenkins wait a minute before proceeding.

After having waited for a minute, we are now ready to query Kubernetes for the IP address of the service that represents the Node.js project. The way we’re going to get the IP address of the Kubernetes service that represents the Node.js application is to use the Kubernetes command kubctl get service.

The kubectl command shown above will display the services running in the cluster. Part of the output will be the external IP address that the cluster exposes for the service, as shown below. This external IP address is the one Jenkins needs in order to run the Ranorex test.

NAME             TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)          AGE
feelingtracker   LoadBalancer   10.43.255.19   35.224.196.118   3000:31891/TCP   1m
Kubernetes       ClusterIP      10.43.240.1    <none>           443/TCP          20m

The way the code in Figure 20 works is to use kubectl to get a list of all the services in the cluster. Then, the output of the kubectl command is sent to the Windows batch command, findstr to filter on a line of code that has the name of the service, feelingtracker. That line of code is sent to a text file on disk, temp.txt. The line of code is used in a Windows batch for loop.

The for loop cycles through the space-delimited words in the line of code stored in the text file to create an array of words. The word at array number 3, which is the fourth element in the array, is the IP address of the service. We already know that the IP address of the Kubernetes service is the fourth element of the array.

Once the IP address is known, we save the information to a predefined Jenkins environment variable property file named env.property as shown in Figure 21 below.

The value that is saved to the environment property file is TEST_IP=!vector[3]! where !vector[3]! contains the IP address of the Kubernetes service representing the Node.js application. After the assignment of the service IP address is made to the environment variable file, the temporary file that contained the line of code created from querying the Kubernetes service is deleted.

Now that we have the IP address of the service in hand, Jenkins can pass the information to the Ranorex test at the command line. Figure 22 below shows the echo command that sends the retrieved IP address to the Jenkins project’s output for display and logging. This is done as a convenience.

At this point, Jenkins knows the location of the Node.js application running on the Internet under Google Cloud. Now the work is to have Jenkins run the test.

Running the Test

Running the actual Ranorex test defined earlier in the Build section of the Jenkins job configuration is declared using the Jenkins Ranorex plugin as shown below in Figure 23.

Running the Ranorex test is accomplished using the standard technique that goes with using the Jenkins Ranorex plugin. As shown in Figure 22 above, the Ranorex test suite file is declared within the working directory location on the Windows Slave where Jenkins stored the code retrieved from GitHub. Also, notice that Jenkins Ranorex plugin is passed the following command line argument:

/pa:webAppUrl=%TEST_IP%:3000

In this command line argument,  %TEST_IP% is the IP address the discovered address of the Kubernetes service representing the Node.js app. This command line argument is the global variable that we defined in Ranorex studio earlier. Ranorex needs this information to find the location of the Node.js application out on the Internet.

Configuring Test Results

The final steps in the Jenkins build job is to declare the location of the test report XML file where Jenkins will store test information. (See Figure 24)

The last step is to remove the Node.js application from the Kubernetes cluster and remove the client-secret.json file that contains the sensitive information required for access to Google Cloud.

Cleaning Up after Testing

We’ll configure Jenkins to clean up upon test completion by using a series of post-build tasks. The first post-build task we’ll configure is to tell Kubernetes to remove the pods that contain the actual Node.js application. These pods exist as a Kubernetes deployment. Thus, we delete the deployment as shown in Figure 25 below.

Next, we tell Kubernetes to delete the service that represents the Node.js application. This is done using the kubectl command as shown in Figure 26 below.

And the last step is to delete the temporary JSON file that contains the Google Cloud access information that Jenkins stored locally to the server’s disk. Figure 27 shows using the Windows batch command, del, to delete the file.

At this point, the test has been run and the demonstration application has been removed from the Kubernetes cluster. The build project and testing is complete.

Putting It All Together

This has been a comprehensive series. Part 1 demonstrated how to use Ranorex and Jenkins to test the GUI of a Node.js application against source code downloaded to the Jenkins Windows Slave server. Part 2 showed you how to containerize the application on the Windows server and then use Docker to invoke the Node.js application and use Ranorex running under Jenkins to test the application. Part 3 provided the instructions for deploying the Node.js application as a container in a Kubernetes cluster in Google Cloud. Finally, this Part 4 showed you how to determine the IP address of the Node.js application running as a Kubernetes service on Google Cloud and then run the Ranorex GUI test against the cloud instance. (See Figure 28.)

We’ve done a lot. It may take a bit of time to absorb all the details presented in the series. Given the increased presence that Jenkins, containers, and Kubernetes have in the enterprise, the hope is that you’ll find this information to be a useful addition to your professional development as a test practitioner using Ranorex and Jenkins with modern cloud technologies.