Iterations counted as testcases in total tally

Ranorex Spy, Recorder, and Studio.
uhmdown
Posts: 54
Joined: Mon Apr 03, 2017 12:00 pm

Iterations counted as testcases in total tally

Post by uhmdown » Mon Apr 03, 2017 12:16 pm

When Data-Binding has been employed, each iteration appears in the Automation Report as a separate test case, and is counted in the total tally.

So if you have 3 testcases, and each run with 30 iterations, then the total count will be 3x30=90 instead of 3.
If one of them fails, then you will have 1 fail out of 90, instead of 1testcase out of 3.

This is pretty inconvenient for us, because we don't always consider an iteration as a separate testcase; for example, a test case with 30 iterations can really just be one testcase with 30 steps.
To have 3 testcases suddenly become 90 makes the reports unreliable for us, because it heavily skews the numbers.


Is there a way to not have the iterations not get counted in the total tally (other than manipulating the ranorex report xml file)?

qwertzu
Posts: 178
Joined: Wed Jan 25, 2017 11:08 am

Re: Iterations counted as testcases in total tally

Post by qwertzu » Tue Apr 04, 2017 1:30 pm

Hi uhmdown,

With Ranorex 7.0, this can be done quite easily because iterations on smart folders are not counted.

So simply create a smart folder under a test case and bind your data on this smart folder. Independent on how many iterations there are, the report will always show if running this single test case was successful or not.

I created a short example with 3 test cases. Each of them have a smart folder as child on which some data is bound. In the 3rd smart folder I added a recording consisting of a report failure action.
After running, the report counts the 2 test cases as successful and 1 as failed.. (See screenshots)

I hope this helps.

regards,

qwertzu
Attachments
Report.png
Report.png (27.14 KiB) Viewed 687 times
TestSuite.png
TestSuite.png (14.75 KiB) Viewed 687 times

uhmdown
Posts: 54
Joined: Mon Apr 03, 2017 12:00 pm

Re: Iterations counted as testcases in total tally

Post by uhmdown » Wed Apr 05, 2017 8:15 am

Yeah, I feel so silly. I saw the update almost immediately after I post, and it solved the iteration thing. Awesome : )