Fast feedback on new code changes marks one of the fundamentals of agile development processes. If you’re using continuous integration and run daily tests, you’ll want to make sure that results are available on a regular basis. Developers can then investigate where the Test Cases failed and react faster on bugs which might have occurred.
When your products grow more complex and developers add features to the mix, they also affect your Test Suite’s complexity. At a certain point the promise to deliver quick results might turn into a challenge. Sounds familiar to you? You might have reached a point where you should check the performance of your Test Suite.
To ensure constant high product quality, Ranorex runs an automated Test Suite to examine its own software builds every night. The massive Test Suite takes around 12 hours to complete. In order to cut down on this, it’s important to find out which parts are having an impact on the performance. In previous versions of Ranorex, one might have had to use the Report to find out about this, but they were limited in detail. Enter the new Performance Tracing feature in Ranorex Studio 9.
Performance Tracing in Ranorex Studio 9
Performance Tracing gives you all the insights you need to understand which parts of your Test Suite are having an impact at the runtime performance. You will receive detailed data reports on every action that occurred during the test run. You can enable Performance Tracing in the Test Suite settings by selecting a tracer. Ranorex Studio 9 comes with multiple tracers that let you specifically look into an area of interest.
Select a tracer you like to investigate in the Test Suite settings
The feature is disabled by default. You’ll notice that there are tracers for various areas of the test run. If you’re not sure yet in which area you would like to investigate, take the “All” tracer as a starting point. Once you have enabled a tracer and run your Test Suite, Ranorex will generate a CSV file containing a detailed overview of timings for each action. This is not limited to Recording actions, but also provides you insights in the overall workings of your test. This also works when you use Ranorex Remote to execute your test on an agent. Usually, you’ll find the file in the Documents folder on your SUT, but you can specify a different location in the Test Suite settings.
Having a first look on a Repository trace log. A search counter of 7 is a good indicator bearing some RanoreXPath optimization potential.
Working with the Data
When you open a CSV file you will find a log of all actions that Ranorex Studio executed during the test run. Depending on the tracer you’ve selected, the data generated might look different. If you choose the Repo tracer, for example, you’ll get an overview of the RanoreXPaths used and the number of attempts to find the said element.
You will also see the duration, in milliseconds, that it took to complete the action. Obviously, you’ll want to sort the list and have a look at the actions taking the most time. However, you can get much more information from the reports. You can see through the actions to spot delays you might have put during test development and forgot to remove or see unnecessary repetitions. Apart from that, you can use analysis tools like Microsoft Power BI to import the raw data and create dashboard-like reports on your test performance.
