Create and execute an ADO pipeline | Ranorex Studio User Guide
Help CenterUser GuideInterfaces and connectivity

Create and execute a pipeline

Pipelines in Azure DevOps are how you build, test, and deploy your code. On this page, you’ll find out how to create, configure, and execute pipelines in Azure DevOps to run your Ranorex Studio solution. For more details about pipelines, please refer to the official Azure Pipelines documentation.

In this chapter

    Create a pipeline

    To create a new pipeline for your Azure DevOps project:

    In your opened project, click Pipelines in the register on the left.

    Click New pipeline.

    Depending on whether you want to specify a pipeline using YAML or on your own:

    Click Azure Repos Git (YAML) (recommended for experts, not covered here).

    Click Use the classic editor.

    Configure as shown below and click Continue.

    In the next step, click Apply for .NET Desktop. This is the framework Ranorex Studio solutions are based on.

    Pipeline overview

    The pipeline overview shows the various components a pipeline consists of and offers many possibilities for configuration. Here we only explain what’s relevant for the Ranorex Studio integration.

    Configuration categories of a pipeline. In the image, Tasks is selected.

    Basic pipeline configuration. Here you can give the pipeline a name, set the agent pool, and set optional parameters.

    Here you can configure the sources (i.e. repositories) the pipeline uses. This corresponds to what we did in Create a pipeline above.

    A pipeline always contains at least one agent job. This is the logical set of tasks that the pipeline carries out.

    An agent job always contains at least one task. Tasks are carried out in the order they appear in under the agent job.

    The configuration window. Configuring one of the above items brings up this window. In the image, the window for configuring the agent job is shown.

    Basic pipeline configuration

    Under Tasks, click Pipeline to bring up the basic pipeline configuration, where you can make the following settings:

    Here you can change the pipeline name.

    Here you select the agent pool for the pipeline. You can use the default local pool or choose from a range of hosted pools that have various images already set up.

    Here you can set optional pipeline parameters. The preset parameter in the image builds the executable from the Visual Studio solution file (.sln file ending).

    Pipeline tasks for Ranorex Studio tests

    For Ranorex Studio tests to be executed correctly in an Azure DevOps pipeline, the agent job must contain the tasks listed below.

    Note icon


    The .NET pipeline already contains various default tasks. Do not delete them.

    Build the Ranorex Studio solution

    The task Visual Studio build builds the Ranorex Studio test solution. It is already included by default in the agent job and functional.


    Change the display name for easier identification.

    Change the platform to x86, if required.

    Execute the build

    To execute the build of the Ranorex Studio solution, you need to add the Command line task to the agent job.

    To do so:

    Click the + symbol next to the agent job.

    In the search bar, enter command line.

    Click Add.

    Note icon


    Make sure the Command line task comes after the Visual Studio build task. Otherwise, execution will fail.


    Change the Display name for easier identification.

    Replace the default script with the following:

    System variable for the current working directory of the Azure DevOps project.

    Folder for the Ranorex Studio solution.

    Folder for the Ranorex Studio project in the solution.

    Default output folder for Visual Studio projects.

    File name of the executable build. You can add further optional arguments to the .exe.


    For a full list of command line arguments and how to use them, refer to

    Ranorex Studio expert > Runtime and remote execution > ⇢ Command line execution


    When running a Ranorex Studio test through an Azure DevOps pipeline, the associated Ranorex Studio report is not saved to the working directory of the build by default. To include it, you have to add a Publish build artifacts task in the agent job. Then:

    Click the Publish build artifacts task.

    Change the display name, if desired.

    Change the directory as shown in the image.

    Change the artifact name, if desired.


    Once a pipeline has finished, you can see the report in the summary and open it from there.

    Click Pipelines > Builds.

    Click Summary.

    Under Build artifacts published, click the item. It will have the name you specified in the task.

    Show test results in the build results

    By default, the results of a Ranorex Studio test are not displayed in Azure DevOps. You can however have them displayed by using the JUnit format of Ranorex Studio reports.

    To do so:

    To the agent job, add a Publish test results task.

    Change the task’s display name, if desired.

    Set the test result format to JUnit.

    As test results files, specify **/*.xml.

    As search folder, specify the current working directory.

    Switch to the Command line task and extend the script as in the image:
    The executable build of the Ranorex Studio solution.

    Argument to create the report in the JUnit format.

    Argument to zip the report.

    Argument to specify a directory and file name for the zipped report.


    For a full list of command line arguments and how to use them, refer to

    Ranorex Studio expert > Runtime and remote execution > ⇢ Command line execution


    Under Pipelines > Builds > Tests, you can see various test details after a build has been executed.

    Execute a pipeline

    To execute a pipeline, you need a  ⇢ configured Azure Pipelines agent.

    Click Pipelines > Builds.

    Click Run.

    In the dialog that appears, review the pipeline settings and click Run.

    Pipeline progress view

    The pipeline will carry out all tasks in the specified order. You can view the progress in Azure DevOps.

    Indicates that a pipeline has been started.

    Start time and elapsed time.

    Completed tasks.

    Current task, including debug info.

    Agent window

    Once you’ve started a pipeline, the agent job is passed to the agent, which then runs it. The agent window will show that a job has been started.

    Ranorex Studio test run

    The agent runs the Ranorex Studio test as if it were started from within Ranorex Studio.

    Pipeline completion

    When a pipeline has completed, it will be indicated in the pipeline overview.

    List of all created pipelines for the current project.

    List of all builds in the selected pipeline, including execution stats (success/failure).


    Click the build to display the results view for it.

    After completing a job, the agent shows the result and then returns to idling.

    Trigger a pipeline automatically

    You can also trigger pipelines automatically. To do so, you need to enable the option Enable continuous integration. Any changes to the Ranorex Studio solution committed and pushed to Git will then trigger the respective pipeline.

    To enable this option:

    Click Pipelines, select the pipeline, and click Edit.

    Click Triggers.

    Check Enable continuous integration.