This blog post will explain the term regression testing and how Ranorex test automation can support in doing functional regression testing. Additionally the blog post will confront you with the problem occurring with regression testing including a strategy to overcome it.
What is Regression Testing
When developing software, every modification of or addition to the existing code might lead to an inoperative state or to an error condition within the existing functionality. These side effects caused by code modifications are called regressions. To determine a regression it’s necessary to re-test already existing and tested parts of a program every time the code has changed.
That’s what regression testing is all about.
How Ranorex can help
Regression testing is not done to expose new defects but to verify the functionality of the existing parts of a program. Testing the same parts of a program after every change in code sounds a little boring and time consuming when doing manually – not to say it is impossible when being performed for every build. For this reason it makes sense to automate regression testing using Ranorex Studio as the ROI of automated regression testing is very high.
A Strategy to Prioritize
Even helping yourself with automated regression tests still does nothing about the fact that there is never enough time to run all test cases.
To minimize risks it is absolutely essential to construct subsets of test cases which have to be executed for every build, test cases which should be executed for every build and test cases which could be executed if there is time left.
After designing your test cases it is time to break them down into the following categories:
- Smoke Tests: The test set which should be executed very first. These tests will determine whether the build is even testable. Smoke tests typically cover a wide but shallow range of functionality.
- High Priority Tests: The test set which is executed most often. These tests ensure that the functionality is stable, intended behaviors and capabilities are working, and important negative tests and boundaries are tested.
- Medium Priority Tests: The set in this category will test the given functional areas and features in a more detailed manner. Additional most of functional specifications are examined including boundary, negative and configuration tests.
- Low Priority Tests: The test set which will be executed least often (e.g. check for error messages, stress tests, UI glitches, etc.).
The most important question that arises is how to figure out which test fits in which category. Following the approach described in the blog post “Rapid Test Case Prioritization” provides a smart and fast way to fulfill this mission.
- Categorize all functional verification tests – also known as happy day scenario tests – as high priority tests.
- Categorize all negative, boundary or validation tests as medium prior tests.
- Categorize all non-functional verification tests as low priority tests.
Now you have a rough division into the three categories. The next steps will refine the selection.
- Divide all functional verification tests into high and low important tests
- Categorize all low important functional test as medium priority tests
- Divide all negative and boundary tests into high and low important tests
- Categorize all high important negative and boundary tests as high priority tests
- Divide all non-functional verification tests into high and low important tests
- Categorize all high important non-functional verification tests as medium priority tests
- Repeat the steps above until there will be no more changes
After doing so, the smoke tests have to be identified.
- Divide all high priority tests into critical and important tests
- Categorize all critical tests as smoke tests.
Using Ranorex to automate your regression tests will in any case dramatically increase the efficiency. However you’ve got to think about how to prioritize your test cases and the given approach might help you in successfully implementing this process.