BUG - Error behavior not working correctly

Bug reports.
krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

BUG - Error behavior not working correctly

Post by krstcs » Mon Jun 01, 2015 3:47 pm

Ranorex 5.3.2
Windows 7

Setting a testcase to use "Continue with sibling" under Error Behavior acts like "Continue with iteration".

Instead of stopping the testcase and moving to the testcase's next sibling, the testcase continues with the next module after the failure. The user guide says that "Continue with sibling" should "in case of an error, the currently running test case should break and the execution should continue with the sibling test case" but this is not happening.

I have created and attached a test suite that demonstrates the issue.
Attachments
ErrorTest.zip
(15.6 KiB) Downloaded 103 times
Shortcuts usually aren't...

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Suite Error Behavior and nested test cases

Post by krstcs » Mon Jun 01, 2015 4:03 pm

I just posted a bug (http://www.ranorex.com/forum/bug-error- ... t8037.html) about error behavior, but while researching that issue I realized that I had a related question about how the errors are supposed to be propagated back up the stack of testcases.

My expectation is that if I set both a child testcase and it's parent testcase to "Continue with Parent" that the child's failure would propagate up to the parent and then up to the grandparent. But I'm not sure this is correct. Would the parent's run continue, or should it stop as well?

Code: Select all

TC_Grandparent -> 'Continue with sibling'
--TC_Parent -> 'Continue with parent'
----TC_Child1 -> 'Continue with parent'
------Module1 -> Fails here...  This should keep Module2 from running and TC_Child2 should not be run because parent is also set to 'Continue with parent'
------Module2 -> Should not be run
----TC_Child2 -> 'Continue with parent'
------Module3
Is this correct?

Not, I'm asking what SHOULD happen, not what does happen. The bug I posted above seems to be causing side-effects with this, so I wanted to know how it is intended to work.
Shortcuts usually aren't...

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: BUG - Error behavior not working correctly

Post by Support Team » Tue Jun 02, 2015 8:04 am

Hello krstcs,

Report.Failure does not actually throw an exception, but rather marks the module as failed. This is by design. Please replace the Report.Failure with an exception or a validation which fails. The error behavior works as expected, please find a sample report attached.
My expectation is that if I set both a child testcase and it's parent testcase to "Continue with Parent" that the child's failure would propagate up to the parent and then up to the grandparent.
This is absolutely correct.

Hope this information helps. Within this scenario the test would continue with "TC_Grandparent2".

Regards,
Robert
Attachments
ErrorTest.zip
(359.08 KiB) Downloaded 63 times
.
Image

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: BUG - Error behavior not working correctly

Post by krstcs » Tue Jun 02, 2015 1:53 pm

Ah! OK, that makes sense. I did think that it threw an exception.

Thanks Robert!
Shortcuts usually aren't...