How to Use Ranorex with Existing Selenium Tests

Introduction

With Ranorex 7, we’ve introduced support for the Selenium WebDriver protocol for web testing. Testers can now use Ranorex to automate their web sites on a wide array of browsers and operating systems, all without the need for plugins or coding.

How it works

The Ranorex WebDriver Integration utilizes the WebDriver Wire Protocol to communicate with browser instances via their individual driver implementations. If you have used Selenium before, you might know that the Selenium API sends JSON data over the WebDriver to execute actions on an endpoint.

Fundamentally, Ranorex does the same, but instead of Selenium APIs, we use our own automation technology and the RanoreXPath. This results in many benefits for you as a tester:

• Object identification via Ranorex Spy is simple and fast
• Instead of coding page objects, you can keep and maintain all of your web site’s elements in the Ranorex Repository
• You can record your tests on your local computer instead of coding them
• We generate a report for you out of the box

The above points show that Ranorex brings a lot of value to your existing Selenium test infrastructure. But what about your existing Selenium tests?

In this blog post, I am going to show you how you can execute an existing Selenium test suite with Ranorex Studio and integrate its results into the Ranorex report. By virtue of Ranorex being a test automation framework, we can leverage the .NET programming language and the new reporting improvements to run an existing test suite either before or after the Ranorex Test Suite and combine the results of both into one report.

• Starting Selenium tests programmed in Java
• Executing a Java test with Ranorex
• Bringing everything together
• Checking the results
• Conclusion
• Resources

Starting Selenium tests programmed in Java/Python/C#

For this example, I’ve created a simple Selenium Test in IntelliJ IDEA utilizing Maven and the Conductor Selenium framework. It consists of two simple test cases and some page objects that automate the Ranorex Web Testing Examples page. This example project will serve as our demo throughout this chapter. Click here to download example project.

The two test cases “testFillForm” and “testValidateForm” are very simple and only serve as a proof of concept. “testFillForm” validates the existence of two input controls and fills them with values, without doing anything else. “testValidateForm” fills the AJAX form on the test page and validates the AJAX response according to the inputs. I’ve included a false positive in order to get a successful and a failed test result.

Reporting test results using Maven

Depending on the environment you are using, there are different ways to export the test results. In my example, I am using the Maven Surefire plugin to generate output for the test results. If you want to integrate Surefire as well, you need to extend the POM file of your project to include the plugin:

    …
    <reporting>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-report-plugin</artifactId>
                <version>2.19.1</version>
            </plugin>
        </plugins>
    </reporting>
</project>

You can configure Surefire to generate reports exactly the way you need. However, in its default configuration, it creates an XML and a TXT file with the results. For this example, we’re sticking tothe defaults, as we’re going to use the XML file later for reporting.

Executing a Java test with Ranorex

After successfully running the two unit test cases in your Java IDE, you can invoke Maven to build the project and run the test cases in the project. To do so, open the Windows command line and run Maven from there.

mvn clean test surefire-report:report

Maven will build the project and run the test cases at the end. Thanks to the Surefire plugin, we can pick up our report from the working directory (under target/surefire-reports/). The plugin should have generated 2 files, an XML file with detailed information as well as a TXT file with a more user-friendly summary. If the command executes as expected, we can use Ranorex to do this for us.

Ranorex can run external applications from within Recordings using the Run Application action, which starts the program in question in parallel. However, to execute our Selenium test suite, we are going to operate from outside the Ranorex Test Suite and want it to run before the Ranorex test starts. That’s why we are going to create a new Code Module in our Solution and name it SeleniumTools. In this Module, we add a new function to run a test synchronously.

using System.Diagnostics;
    public void RunTestSynchronized(string command, string args, string workdir)
    {
        Report.Info("Selenium", "Starting " + command + " with Arguments " + args + " in " + workdir);
        ProcessStartInfo psi = new ProcessStartInfo();
        psi.FileName = command;
        psi.Arguments = args;
        psi.WorkingDirectory = workdir;

        try
        {
            var proc = Process.Start(psi);
            proc.WaitForExit();
            Report.Info("Selenium", "Done!");
        }
        catch(Exception e)
        {
            Report.Error("Selenium", "Could not start Selenium Testn" + e.Message);
        }
    }

The above code causes Ranorex to launch the application supplied via command with the arguments in args and run it in the working directory workdir. Furthermore, Ranorex will wait until the process completes. The reporting engine is invoked as well so that we have some information in the Report about the externally started application.

Parsing the Results

Now that we can successfully start our external environment with our existing Selenium test, we also want to include its results in our Test Suite. As stated before, Apache Surefire generates a report that we can use in order to verify the success of our test cases. Let’s have a look at the structure of the XML that Surefire generates:

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite tests="2" failures="0" name="com.ranorex.selenium.TestPageTest" time="8.794" errors="0" skipped="0">
    <properties>
        <property name="java.runtime.name" value="Java(TM) SE Runtime Environment"/>
        ...
    </properties>

<testcase classname=”com.ranorex.selenium.TestPageTest” name=”testFillForm” time=”1.62″/> <testcase classname=”com.ranorex.selenium.TestPageTest” name=”testValidateForm” time=”7.174″/> </testsuite>

The above XML is from the report generated by the recording we just executed. We can see the following interesting parts:

• The number of tests in the test suite
• The number of failed tests in the test suite
• The number of errors that occurred during execution
• The number of skipped tests

The individual test cases are listed below the properties block. Both of them show each test case’s name and package. If a test case fails, it will look like this:

  <testcase classname="com.ranorex.selenium.TestPageTest" name="testValidateForm" time="7.676">
    <failure message="Text does not match! [expected: Array
(
    [value1] =&gt; Hi from
    [value2] =&gt; Ranorex
    [checkbox2] =&gt; true
    [color2] =&gt; bluex
)] [actual: Array
(
    [value1] =&gt; Hi from
    [value2] =&gt; Ranorex
    [checkbox2] =&gt; true
    [color2] =&gt; blue
)]" type="java.lang.AssertionError">java.lang.AssertionError: Text does not match! [expected: Array
(
    [value1] =&gt; Hi from
    [value2] =&gt; Ranorex
    [checkbox2] =&gt; true
    [color2] =&gt; bluex
)] [actual: Array
(
    [value1] =&gt; Hi from
    [value2] =&gt; Ranorex
    [checkbox2] =&gt; true
    [color2] =&gt; blue
)]
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at io.ddavison.conductor.Locomotive.validateText(Locomotive.java:668)
        at com.ranorex.selenium.TestPageTest.testValidateForm(TestPageTest.java:43)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
        at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
</failure>
    <system-err>Starting ChromeDriver 2.27.440174 (e97a722caafc2d3a8b807ee115bfb307f7d2cfd9) on port 10532
Only local connections are allowed.
</system-err>
   </testcase>

Above you can see my false positive. The values shown do not match – intentionally. That’s quite a lot of useful information that we want to see in our Ranorex report as well, since we want to give as much information to our developers as possible. In order to do so, we need to parse the XML and invoke the Reporting API to add that information.

Ranorex Test Suites and Test Cases

In order to integrate the Surefire results meaningfully into the Ranorex report, you first need to understand how Ranorex Test Suites correlate with the reporting engine. A Ranorex Test Suite starts the reporting autonomously and assumes all test cases are provided exclusively by itself.. This naturally does not play well with outside test cases. In order to overcome this, you need to decide whether you want your existing Selenium tests executed before or after the Test Suite, and have them inserted accordinglyinto the report. In the following example, we are going to invoke the reporting engine manually and insert our Selenium test results into the “Before Test Suite” block.

We start by adding a new function to our Code Module. This method will load the XML file once the Selenium test suite completes and inserts the results from it into the Ranorex report.

public void ParseExtTestResults(string workdir, string fileName)
        {
            XmlDocument xdoc = new XmlDocument();

            try
            {
                xdoc.Load(workdir + fileName);

                XmlNode xtestsuite = xdoc.SelectSingleNode("/testsuite");
                XmlNodeList testcases = xdoc.SelectNodes("/testsuite/testcase");

                string sHead = string.Format("External Selenium Test completed. {0} out of {1} test cases successful",
                                             Int32.Parse(xtestsuite.Attributes["tests"].Value) - Int32.Parse(xtestsuite.Attributes["failures"].Value),
                                             Int32.Parse(xtestsuite.Attributes["tests"].Value));

                TestReport.BeginTestEntryContainer(1, "Selenium Test Suite (external)",  ActivityExecType.Execute, TestEntryActivityType.TestCase);
                TestReport.BeginSmartFolderContainer("Selenium", sHead);

                Report.Info("Selenium", "Parsing Results...");

                foreach(XmlNode n in testcases)
                {
                    Ranorex.Core.Reporting.TestReport.BeginTestCaseContainer(n.Attributes["classname"].Value + "." + n.Attributes["name"].Value);
                    Ranorex.Core.Reporting.TestReport.BeginTestModule("Module " + n.Attributes["name"].Value);

                    if(n.SelectSingleNode("failure") != null)
                    {
                        Report.Failure("Selenium", "Test case " + n.Attributes["name"].Value + " (" + n.Attributes["classname"].Value + ") n" +
                                       n.SelectSingleNode("failure").Attributes["message"].Value);

                        Ranorex.Core.Reporting.TestReport.EndTestModule();
                        Ranorex.Core.Reporting.TestReport.EndTestCaseContainer(TestResult.Failed);
                        continue;
                    }

                    Report.Success("Selenium", "Test case " + n.Attributes["name"].Value + " (" + n.Attributes["classname"].Value + ")");

                    Ranorex.Core.Reporting.TestReport.EndTestModule();
                    Ranorex.Core.Reporting.TestReport.EndTestCaseContainer(TestResult.Passed);
                }

                TestReport.EndTestCaseContainer();
                TestReport.EndTestEntryContainer();
            }
            catch(Exception e)
            {
                Report.Error("Selenium", "Error reading Resultsn"+e.Message);
            }
        }

Bringing everything together

Now we have everything we need to run our Selenium test suite from within Ranorex. In the Program.cs file, create an instance of your SeleniumTools class and call the two newly created functions before the TestSuiteRunner call.

        [STAThread]
        public static int Main(string[] args)
        {
            // Uncomment the following 2 lines if you want to automate Windows apps
            // by starting the test executable directly
            //if (Util.IsRestartRequiredForWinAppAccess)
            //    return Util.RestartWithUiAccess();

            Keyboard.AbortKey = System.Windows.Forms.Keys.Pause;
            int error = 0;

            SeleniumTools st = new SeleniumTools();

            st.RunTestSynchronized("mvn",
                "clean test surefire-report:report",
                @"C:UserscopresnikIdeaProjectsselenium");

            st.ParseExtTestResults(@"C:UserscopresnikIdeaProjectsselenium",
                @"targetsurefire-reportsTEST-com.ranorex.selenium.TestPageTest.xml");

            try
            {
                error = TestSuiteRunner.Run(typeof(Program), Environment.CommandLine);
            }
            catch (Exception e)
            {
                Report.Error("Unexpected exception occurred: " + e.ToString());
                error = -1;
            }
            return error;
        }

Checking the results

When you hit the Run button or press F5, Ranorex will now first call the mvn build and execute the unit tests. Using the Surefire plugin, Maven will generate a report which will then be parsed and inserted into your report.

Your report should look like this:

Note how the external test cases are organized in a new Smart Folder and are counted in the report. Our false positives even report what’s wrong with the faulty test case. The Ranorex test case is executed afterwards.

Conclusion

Instead of recreating every single test case you’ve designed with Selenium in the past, you can now trigger them easily in your Ranorex environment. From there you can continue to create new and additional test cases with Ranorex Studio. Since maintenance tasks are much more efficient when using the Ranorex Repository and Spy, you can cut down on maintenance efforts even more without losing the functionality of your previous work.

Resources

You can find and download the SeleniumTools class and its functions from our github repository.
The example IntelliJ project is available for download.
The example Ranorex Solution is available for download.

More about the Selenium integration Download Trial