How do I set the current test case status to "Skipped"

Class library usage, coding and language questions.
tvu
Posts: 195
Joined: Tue Apr 07, 2015 10:47 pm

How do I set the current test case status to "Skipped"

Post by tvu » Wed Aug 31, 2016 5:41 pm

Hey guys,

tldr:
How do I end a test case immediately and set the status as skip?

Full Description:
Often time, we have a feature that we are testing in QA where we constantly toggle on and off throughout the day. We would have several test cases that would be design to test this feature. We have separate run configurations for when this feature is on and off, but sometime the timing doesn't match up. i.e, I have the test suite schedule to run in the evenings with the feature on, but then it gets toggled off beforehand.

What I would like to do is determine the feature status at run time and set a global parameter. I would check the global parameter at the setup phase for each test case related to this feature. If it's disabled, I want to skip the test case and mark it as skipped.

I know how to disable a test case if it hasn't started yet, but I don't know how to end the current test case and then set the status as skip. I would like to immediately end the test case. I don't want the rest of the Setup phase to run. I don't want the Teardown phase to run.

Thanks!

PS: I think I can use this to stop the test case execution. Still not sure on how to mark it as skipped.
http://www.ranorex.com/forum/how-to-sto ... tml#p24582

tvu
Posts: 195
Joined: Tue Apr 07, 2015 10:47 pm

Re: How do I set the current test case status to "Skipped"

Post by tvu » Fri Sep 02, 2016 5:58 pm

So I am able to disable all modules and child test cases with this bit of code:

Code: Select all

if (boolSkipTestCase.ToLower() == "true")
            {
            	var currentTestCase = (TestCaseNode)TestCaseNode.Current;
            	Report.Warn("Skipping current test case: " + currentTestCase.DisplayName);
            	
            	// Disabling all modules. Including those in Setup and Tear Down Phases that are proceeding this module.
            	foreach (TestModuleLeaf childModule in currentTestCase.AllModules)
            	{
            		childModule.Enabled = false;
            		Report.Info("Disabiling module: " + childModule.Name);
            	}
            	
            	// Disabling all child test cases
            	foreach(TestCaseNode childTestCase in currentTestCase.GetDescendantTestCases())
            	{
            		childTestCase.Checked = false;
            		Report.Info("Disabiling child test case: " + childTestCase.DisplayName);
            	}
            }
I have this in a code module and place it at the very beginning of the setup phase for my parent test case. I bind the boolSkipTestCase variable to a global parameter.

Unfortunately, I still can't figure out how to set the result of this test case as "skipped". Can someone point me in the right direction?

Thanks.

johnzer
Posts: 16
Joined: Thu Aug 18, 2016 9:27 pm

Re: How do I set the current test case status to "Skipped"

Post by johnzer » Fri Sep 02, 2016 6:19 pm

Hey tvu,

I'm a little unclear as to what it is just want to do.

You want to disable all child modules of the current running test, then end the current test, and manually mark it as skipped?

Have you looked at the EndTestCase() method in the TestReport class?

http://www.ranorex.com/Documentation/Ra ... Report.htm

tvu
Posts: 195
Joined: Tue Apr 07, 2015 10:47 pm

Re: How do I set the current test case status to "Skipped"

Post by tvu » Fri Sep 02, 2016 7:05 pm

Hi johnzer,

Yes, basically I want to end the test case immediately and mark it a skipped. I haven't looked into the TestReport class. I'll give it a try.

Thanks for the tip.

johnzer
Posts: 16
Joined: Thu Aug 18, 2016 9:27 pm

Re: How do I set the current test case status to "Skipped"

Post by johnzer » Fri Sep 02, 2016 7:38 pm

tvu wrote:Hi johnzer,

Yes, basically I want to end the test case immediately and mark it a skipped. I haven't looked into the TestReport class. I'll give it a try.

Thanks for the tip.
Cool! Then you should be able to do it using the TestReport class, then. :D

krstcs
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: How do I set the current test case status to "Skipped"

Post by krstcs » Fri Sep 02, 2016 8:29 pm

Ranorex already has this capability built in, although it won't mark the test case as skipped, just as failed, which skips the rest of the test case anyway.

You just need to use the Error Handling settings on each test case (right-click the test case) and set it to "Continue with sibling".

Since you already know that if a module fails in the test case that Ranorex will skip the rest of that test case, you don't need to mark it as skipped, and it should be marked as failure anyway since you never expect anything other than success or failure if the tests are designed correctly.
Shortcuts usually aren't...

tvu
Posts: 195
Joined: Tue Apr 07, 2015 10:47 pm

Re: How do I set the current test case status to "Skipped"

Post by tvu » Fri Sep 02, 2016 11:58 pm

Hi krstcs,

That would give me a failure that I would then have to analyze to figure out that it's a false failure. Ideally, I only want to run the test case if the feature is enabled. I know I can write a module that can disable test cases that hasn't started yet by doing the following:
TestSuite.Current.GetTestCase(TESTCASENAME).checked = false;

But, that would force me to maintain a list of test case name. I was trying to be a little more dynamic. I was willing to compromise and get a "skipped" test reported. If I see that reported than I automatically know the reason.


Hi johnzer,

Calling EndTestCase(TestResult.Skipped) seems to just mark the test case as skip, but still runs everything anyways. Seems like I might have to disable everything and then call EndTestCase. If I do this in the Setup phase, it will only mark the Setup phase as "Skipped". I have to run this as the first module in the Parent test case and have nothing in the Parent's Setup phase in order to get the desired result.

I am pretty sure I am doing this wrong.

krstcs
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: How do I set the current test case status to "Skipped"

Post by krstcs » Tue Sep 06, 2016 3:22 pm

You should never have "false failures". If the test is setup correctly, then any failure should mean there is a bug in the SUT.

You should know exactly what the expected outcome is for every test case, so you should be able to code for it in each situation. Positive tests should be testing a positive SUT outcome and should pass if that happens. Negative tests should be testing a negative SUT outcome (error is thrown, etc.) and should pass if that happens.

If that is the case, then, again, there is no need to mark anything as skipped.

I would suggest that you may want to re-examine your testing methodology and make sure that you don't have any false negatives and that all testing has a known expected result given the test script and data combination.
Shortcuts usually aren't...