Test Case Management with TestRail and Ranorex Webtestit

Test cases are an essential part of any software testing process. They help us determine if features of an application or system under test work correctly. The efficiency of testing depends on how well we manage our test cases.

A test case can have the following properties:

• Test case ID — Usually a numerical value to identify a unique test case
• Test case summary — The summary/description of the test case, for example, “Check customer login with valid data”
• Preconditions — Any prerequisites that must be fulfilled before the execution of the test case
• Test data — Variables and their values that are to be used while conducting the test, for example, a user ID and password
• Test steps — Step-by-step procedure to execute the test, e.g. open a website, enter user ID and password, click the ‘Submit’ button, etc.
• Expected result — The expected result of the test, e.g. User is successfully logged into an application
• Actual result — The actual result of the test, e.g. login was successful/unsuccessful
• Pass/Fail — Marks whether the test case passed or failed
• Comments — Any comments on the test case or test execution.

Of course, additional properties may be included depending on the policy or method used by the particular team to create test cases.

Different policies may also determine how teams should document, execute and track test cases, in a process known as test case management.

Test case management is essential for the efficient testing process because it helps testers, developers, managers, and other team members to keep track of test cases, as well as test strategies, test plans, and test runs in detail.
Failure to document test cases may result in test steps being unnecessarily repeated or, worse, missed completely.
There are a number of ways to manage test cases, and some are more efficient than others. For example, we can use spreadsheets to add test case properties manually or track everything we test in a bug tracking tool, such as Jira, using a custom issue type.

But these two methods include some problems. Using Excel or Google Docs to manage your test cases may work on a small scale. However, as your team or app grows, this approach can become troublesome. The spreadsheet document may get too long and become hard to filter/search. Also, there would be no execution history (pass, pass, fail) and you may find reporting too cumbersome.

The problem with issue tracking tools such as JIRA is that they can be configured to support test cases, but they do not provide testing-specific features, such as reports, execution history, etc.
Furthermore, using either of these methods means that we need to create and track our test cases manually, which will definitively slow us down

This is why many companies resort to specialized test case management tools such as TestRail to track, maintain, and analyze their test cases.

About TestRail

TestRail is a web-based tool that helps developers and QA teams to easily manage and track their manual and automated test cases and test runs.

The tool covers important aspects of software testing, ranging from simple tracking of each individual test status to detailed planning and coordination of the entire testing process.

Thanks to informative dashboards, metrics, activity reports and email notifications, TestRail provides real-time insights into testing progress.

One of the greatest strengths of TestRail is the ability to integrate with many popular project management and issue tracking tools, such as Atlassian Jira, Bugzilla, TFS, GitHub, Visual Studio, FogBugz, Assembla, as well as test automation tools, including Ranorex Webtestit.

You can check the official documentation to see how each of the supported tools integrates with TestRail.

Integration with Ranorex Webtestit

In this article, we’ll use TestRail integration with Ranorex Webtestit to explain how to easily manage your test cases.
This integration will allow us to import TestRail projects into Ranorex Webtestit and then export test reports back to TestRail once the test execution in Ranorex Webtestit is completed.

Enabling TestRail API is the prerequisite for this integration, so the first step will be to open TestRail, then go to ‘Administration’ > ‘Site Settings’ > ‘API’ and check ‘Enable API. ‘ We’ll click the ‘Save Settings’ button to complete the action.

Also, as we are going to import some TestRail projects into Ranorex Webtestit, we should first create them (if we have not created any).

In the ‘Administration’ section, select ‘Projects’ and then click the ‘+ Add Project’ button.

On the ‘Add Project’ page, we can add project details, such as the project’s name, user roles that can access the project, defects, references, etc.

After we’ve created a project in TestRail, we’ll be able to manage it from the TestRail Dashboard.

Adding test cases to a project will be the first step. To do that, go to the ‘Test Cases’ tab and click the ‘+ Add Test Case’ button.

Of course, we can create multiple test cases in TestRail projects. For each test case, we can add details, such as title, type, priority, preconditions, test steps, etc.

After we’ve added all test cases to our project, we can prepare things for a test run by clicking the ‘+ Add Test Run’ button.

TestRail allows us to add additional details for our test run such as name, milestone, to whom it will be assigned, etc. Also, here we can select test cases that will be included in a test run, either all of them or specific cases.

After setting all the required prerequisites in TestRail, we can open a new project in Ranorex Webtestit and enable integration with TestRail.

To do so, we will open preferences dialog by clicking on ‘Ranorex Webtestit’ > ‘Preferences’ > ‘TestRail integration.’

In the Preferences dialog, we will enable the integration and then enter our TestRail’s profile URL available in the following form: https://yourHostURL.testrail.io

Of course, you need to replace ‘yourHostURL’ with the web address of your TestRail account.
After clicking the ‘Connect’ button, we will be prompted to enter our TestRail credentials, e.g. our email and password. If the URL and credentials are valid, the green ‘Connected’ status will appear in the dialog.

We can also specify the ‘Project ID’ in the Preference dialog, but we can leave this field blank and choose the project later while importing TestRail projects.

Click the ‘Save and Close’ button to close the Preference dialog and complete this step.

After saving the changes, the ‘Import from TestRail’ icon should be visible above the project tree. This icon is only available if the TestRail account is connected with Ranorex Webtestit.

Clicking on the icon will open the import dialog where we can select the project to be imported from a dropdown list. The tests are organized in the project’s suites, sections, and sub-sections. We can also use the filter to only show types of test cases that are automated.

The imported test file will contain information about the test case and test suite IDs, as well as a direct link for opening the test case in a browser.

If you’ve added test steps while creating a test case in TestRail, they will be formatted and displayed as comments in the test file.

We’ll then create the test steps in the imported test file, and then add an endpoint to start the test execution.

Once the test execution is complete, we’ll have an option to export the results back to TestRail. To do that, simply open the test report and click the ‘Export to TestRail’ icon.

You can customize the export format for the test run in a new dialog. Choose the format you want, hit the ‘Upload’ button and the selected test result will be uploaded to your TestRail account.

We can jump to the test run using the ‘View in TestRail dashboard’ link in the export dialog…

… or directly from the report.

The imported tests will be available in the ‘Test Runs & Results’ tab on the TestRail dashboard.

Clicking on each test will display the report with all the important details about a particular test run.

TestRail and Ranorex Webtestit can also integrate with Jira, which allows us to directly create bug tickets if any of the test runs failed or produced errors.

TestRail also lets us customize pretty much everything in it, including test case fields (properties), case types, templates, priorities, etc.

QA teams may find TestRail ideal for their workflow as it supports project collaboration between multiple users with different roles, with an option to add more custom roles and permissions.

Conclusion

The efficiency of testing depends on how quickly and easily we can create test cases, run tests, and track results. In this article, we showed how easily you can manage multiple test cases and test runs if you integrate TestRail with your test automation tool.

Here, we explained the integration with Ranorex Webtestit, which also provides a lot of features and integrations essential for seamless testing.