Challenge
A broad range of technologies created a demanding testing environment, including some applications that could only be tested through the GUI.
Broad range of technologies
Applications include C++ modules, Java modules using both Swing and Eclipse RCP, and a few web modules implemented with Angular. There are also several GUI technologies like QT, as well as older technologies such as the ISA framework or VX/VA. The testing strategy needed to cover all of these technologies.
Frequent third-party code updates
Increasingly, the team found that companies that deliver third-party modules were updating their versions (i.e. JBOSS) more often than than the in-house team was updating. As a result, the team needed a way to quickly perform integration testing on these third-party modules before releasing the new version to their customers.
Time-consuming smoke testing
The team wanted to do a smoke test for all modules for every software version, to ensure basic functionality such as the application opening, libraries loading, successful printing, etc. However, in some cases, this might mean hundreds of modules: a time-consuming and cumbersome process.
Result
Development teams can release more often and identify defects earlier, due to more frequent testing and shorter test cycles.
To identify the right test automation solution for their situation, the team initially conducted a series of web searches to identify candidates. “We chose Ranorex as the first one, and then we spent one or two weeks trying it out: accessing all the different technologies, end-to-end, to see how it runs, and in what environments. [We investigated] how it behaves, what happens if we change the O/S from Windows 10, to make sure we can develop in one version and that doesn’t break the solution. We tested all this out, and then calculated the price, and checked again if we could find another solution which could do the same things. But we didn’t find a better solution. Then we asked for reference customers, and we found one in Berlin. We spent one day with them and this was also very helpful.”
Although feature releases happen on a quarterly schedule based on customer preference, there may be daily releases to the internal environment. Now, the team saves significant time by automatically running smoke tests of all modules in a release — even when this may include hundreds of modules.
Recommendations
When it comes to adopting test automation, Ms. Reimer made the following suggestions: “One thing to know that it takes time — maybe more than you think in the beginning. You have to know that you need to really get into it and have time to implement and maintain. You need to think about the process — what you want to do with the automation, when to run it, what to do if there is a finding, what if the solution is not running anymore. It takes time for the company to get used to the new process.”
