How to change overall result after Test from failure to warning?

Best practices, code snippets for common functionality, examples, and guidelines.
BCTest
Posts: 88
Joined: Tue Jun 03, 2014 10:15 am
Location: Hamburg, Germany

How to change overall result after Test from failure to warning?

Post by BCTest » Fri Aug 31, 2018 7:23 am

Hi,

a weird question -I know- but how can we change the overall result after a test in the report?
In one of our tests we let Ranorex validate the interaction between our programme and a third-party-programme. Sometimes this third-party-programme is not working properly for many reasons. In this case we want our test to stop but not to fail. The overall result of the test should be a warning not a failure: Is it posible to to this with Ranorex? And if it is possible: How can we manage this?

Regards,
BcTest.

User avatar
RobinHood42
Posts: 238
Joined: Fri Jan 09, 2015 3:24 pm

Re: How to change overall result after Test from failure to warning?

Post by RobinHood42 » Fri Aug 31, 2018 8:46 am

Hi,
Sometimes this third-party-programme is not working properly for many reasons
How would you know the third party tool isn't working? I guess that elements just aren't found and the automation fails. In that case, there is no way to neglect the failure of the test.

Cheers,
Robin :mrgreen:

BCTest
Posts: 88
Joined: Tue Jun 03, 2014 10:15 am
Location: Hamburg, Germany

Re: How to change overall result after Test from failure to warning?

Post by BCTest » Fri Aug 31, 2018 11:18 am

Hi Robin,

I agree: We won't know which programme will cause the failure.
But just for your interest:
We use the same test with different test parameters. E.g. some tests will run in german environment, some will run in austrian environment. So this particular test will run five times each day with different parameters in different environments. If only one (Or two? Or three? Or Four?) test fail the thrid-party-programme could be the failure. If all tests fail it could be our programme can cause it. Maybe :D .

Regards,
BcTest

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3922
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: How to change overall result after Test from failure to warning?

Post by odklizec » Mon Sep 03, 2018 1:50 pm

Hi,
qwertzu wrote:
Mon Sep 03, 2018 1:19 pm
There is no setting which will give you a "green" report even if elements could not be found.
A small correction...there actually is an option, which will make the report green, even if an element is not found. It's just impractical to use in wide scale, because it must be set per individual recording line.
optionalstep.png
optionalstep.png (20.93 KiB) Viewed 94 times
Setting option "Continue on fail" = True will make the step optional and prevent the test to fail in case of failure. I would suggest to use this option with caution and only if really necessary.

Of course, the same can be achieved in code, by implementing Try...Catch block, which is basically what Ranorex does for recording steps. This is how the above optional line looks in code...

Code: Select all

	try {
		Report.Log(ReportLevel.Info, "Wait", "(Optional Action)\r\nWaiting 5s to exist. Associated repository item: 'Notepad'", repo.Notepad.SelfInfo, new ActionTimeout(5000), new RecordItemIndex(1));
		repo.Notepad.SelfInfo.WaitForExists(5000);
	} catch(Exception ex) { Report.Log(ReportLevel.Warn, "Module", "(Optional Action) " + ex.Message, new RecordItemIndex(1)); }
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

qwertzu
Posts: 178
Joined: Wed Jan 25, 2017 11:08 am

Re: How to change overall result after Test from failure to warning?

Post by qwertzu » Mon Sep 03, 2018 2:13 pm

hey,
of course, pavel you're right on that point, but still the action itself won't be green.
That's what I meant. It will be shown as a warning.
regards, qwertzu

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3922
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: How to change overall result after Test from failure to warning?

Post by odklizec » Mon Sep 03, 2018 2:21 pm

Yeah, you are right qwertzu. There will be a warning in report (unless he decides to create userdode-based Try...Catch block, with ReportLevel.Info). But the report will stay green, which I think is important for him? But as mentioned, 'optional' steps should be used with caution and will definitely not help in case of randomly failing steps.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration