What to Automate
In theory, any software test can be automated. The question is whether a particular test will cost more to develop and maintain than it will save in testing. To get the best return on your effort, focus your automation strategy on test cases that meet one or more of the following criteria:
Tests for stable features
Load tests are simply a variation on data-driven testing, where the goal is to test the response of the system to a simulated demand. Combine a data-driven test case with a tool that can execute the test in parallel or distribute it on a grid to simulate the desired load.
What is Difficult to Automate
The following types of test cases are more difficult to automate. That doesn’t mean that they shouldn’t be automated – only that these test cases will have a higher cost in terms of time and effort to automate. Whether a particular test case will be challenging to automate varies depending on the technology basis for the AUT. If you are evaluating an automation tool or doing a Proof of Concept, be sure that you understand how the tool can help you overcome these difficult-to-automate scenarios.
Waiting for events
Certain aspects of web applications
There are aspects of web applications that present unique challenges to automation. One of the primary issues is recognizing UI elements with dynamic IDs. Ranorex provides “weight rules” to tweak the RanoreXPath for specific types of elements, which helps ensure robust object recognition even on dynamic IDs. Other challenges in automating web applications include switching between multiple windows and automating iframes — especially those with cross-domain content. Ranorex Studio can detect and automate objects inside cross-domain iframes, even when web security is enabled.
Certain aspects of mobile applications
What You Shouldn’t Automate
There are some types of tests where automation may not be possible or advised. This includes any test where the time and effort required to automate the test exceeds the potential savings. Plan to perform these types of tests manually.
Tests with unpredictable results
Features that resist automation
Native O/S features on mobile devices
To ensure that you achieve your automation goals, focus your automation efforts on the right test cases. And be sure to build in time for exploratory testing and UX/usability testing – by their nature, these types of tests can’t and shouldn’t be automated. To help determine whether or not to automate a particular test case, you can use the Test Case ROI Calculator spreadsheet. This simple spreadsheet compares the estimated time and costs to automate a test case vs. the time and costs to execute the same test case manually; it is not designed to determine the ROI of a test automation project.