Ranorex Forum

Hi,

I've attached 3 images for a test report that I generated with Ranorex. As you'll see in the Error Report file the "failed" test count is 6; however if you look at the other two images you'll see that only 2 recordings failed. We are trying to use the failed count as a metric for the current stability of our product, however this is not possible with the current incorrect failure tally method.
So I have two questions. One, could you explain how this failure count is reached specifically what constitutes a failure in this report. Two, is there any way we can change how this failure tally is generated? We'd like it to only count a recording as a failure ( test cases and iterations should not count as a failure, since the recording underneath them already count ). We are using Ranorex 3.3.3.

Thank you
-Nick

Hello,

If a test case fails the failure count will be incremented.
There could be many reasons for failing a test: e.g. a validation fails, elements are not found etc.

Unfortunately, there is no way to change this failure count since it is implemented in Ranorex.Core.
We will discuss this feature request with our developers.

You could customize your report.
Please take a look at section Customizing Ranorex Reports in our blog.

Regards,
Markus (T)

Markus,

How would customizing my report help solve the issue with the failure count? Is it clear how the error count is being improperly reported in the screenshots I attached? I understand there may not be a way to change the failure count, however I am reporting this as a bug in the ranorex software not a feature request for modifying failure counts. I'd like to see the correct error count on my ranorex reports, which in my opinion should be 2 (not 6) for the TacViewSymbology test suite report I attached.

Hello,

You could customize your reports in order to get such a failure count but it would result in a big effort.

Each Test Case is designed as one unit which can fail or pass.
If one recording failed within a Test Case the whole Test Case would fail.

In your case, you have parent and child Test Cases. If one child Test Case failed the parent Test Case would fail as well. Both failed Test Cases (parent and child) would increment the failure count.

Regards,
Markus (T)

Markus,

Thank you for the explanation. I'd like to report the parent/child failure as a bug. I can't think of a scenario where we would ever want to count that as two separate failures. It makes the parent/child feature of test cases not work for organizing tests appropriately.

Thanks,
-Nick

Hello,

Thank you for your information.
We will discuss this internally.

Regards,
Markus (T)