Hey guys,
tldr:
How do I end a test case immediately and set the status as skip?
Full Description:
Often time, we have a feature that we are testing in QA where we constantly toggle on and off throughout the day. We would have several test cases that would be design to test this feature. We have separate run configurations for when this feature is on and off, but sometime the timing doesn't match up. i.e, I have the test suite schedule to run in the evenings with the feature on, but then it gets toggled off beforehand.
What I would like to do is determine the feature status at run time and set a global parameter. I would check the global parameter at the setup phase for each test case related to this feature. If it's disabled, I want to skip the test case and mark it as skipped.
I know how to disable a test case if it hasn't started yet, but I don't know how to end the current test case and then set the status as skip. I would like to immediately end the test case. I don't want the rest of the Setup phase to run. I don't want the Teardown phase to run.
Thanks!
PS: I think I can use this to stop the test case execution. Still not sure on how to mark it as skipped.
http://www.ranorex.com/forum/how-to-sto ... tml#p24582
How do I set the current test case status to "Skipped"
Re: How do I set the current test case status to "Skipped"
So I am able to disable all modules and child test cases with this bit of code:
I have this in a code module and place it at the very beginning of the setup phase for my parent test case. I bind the boolSkipTestCase variable to a global parameter.
Unfortunately, I still can't figure out how to set the result of this test case as "skipped". Can someone point me in the right direction?
Thanks.
Code: Select all
if (boolSkipTestCase.ToLower() == "true")
{
var currentTestCase = (TestCaseNode)TestCaseNode.Current;
Report.Warn("Skipping current test case: " + currentTestCase.DisplayName);
// Disabling all modules. Including those in Setup and Tear Down Phases that are proceeding this module.
foreach (TestModuleLeaf childModule in currentTestCase.AllModules)
{
childModule.Enabled = false;
Report.Info("Disabiling module: " + childModule.Name);
}
// Disabling all child test cases
foreach(TestCaseNode childTestCase in currentTestCase.GetDescendantTestCases())
{
childTestCase.Checked = false;
Report.Info("Disabiling child test case: " + childTestCase.DisplayName);
}
}
Unfortunately, I still can't figure out how to set the result of this test case as "skipped". Can someone point me in the right direction?
Thanks.
Re: How do I set the current test case status to "Skipped"
Hey tvu,
I'm a little unclear as to what it is just want to do.
You want to disable all child modules of the current running test, then end the current test, and manually mark it as skipped?
Have you looked at the EndTestCase() method in the TestReport class?
http://www.ranorex.com/Documentation/Ra ... Report.htm
I'm a little unclear as to what it is just want to do.
You want to disable all child modules of the current running test, then end the current test, and manually mark it as skipped?
Have you looked at the EndTestCase() method in the TestReport class?
http://www.ranorex.com/Documentation/Ra ... Report.htm
Re: How do I set the current test case status to "Skipped"
Hi johnzer,
Yes, basically I want to end the test case immediately and mark it a skipped. I haven't looked into the TestReport class. I'll give it a try.
Thanks for the tip.
Yes, basically I want to end the test case immediately and mark it a skipped. I haven't looked into the TestReport class. I'll give it a try.
Thanks for the tip.
Re: How do I set the current test case status to "Skipped"
Cool! Then you should be able to do it using the TestReport class, then.tvu wrote:Hi johnzer,
Yes, basically I want to end the test case immediately and mark it a skipped. I haven't looked into the TestReport class. I'll give it a try.
Thanks for the tip.
Re: How do I set the current test case status to "Skipped"
Ranorex already has this capability built in, although it won't mark the test case as skipped, just as failed, which skips the rest of the test case anyway.
You just need to use the Error Handling settings on each test case (right-click the test case) and set it to "Continue with sibling".
Since you already know that if a module fails in the test case that Ranorex will skip the rest of that test case, you don't need to mark it as skipped, and it should be marked as failure anyway since you never expect anything other than success or failure if the tests are designed correctly.
You just need to use the Error Handling settings on each test case (right-click the test case) and set it to "Continue with sibling".
Since you already know that if a module fails in the test case that Ranorex will skip the rest of that test case, you don't need to mark it as skipped, and it should be marked as failure anyway since you never expect anything other than success or failure if the tests are designed correctly.
Shortcuts usually aren't...
Re: How do I set the current test case status to "Skipped"
Hi krstcs,
That would give me a failure that I would then have to analyze to figure out that it's a false failure. Ideally, I only want to run the test case if the feature is enabled. I know I can write a module that can disable test cases that hasn't started yet by doing the following:
TestSuite.Current.GetTestCase(TESTCASENAME).checked = false;
But, that would force me to maintain a list of test case name. I was trying to be a little more dynamic. I was willing to compromise and get a "skipped" test reported. If I see that reported than I automatically know the reason.
Hi johnzer,
Calling EndTestCase(TestResult.Skipped) seems to just mark the test case as skip, but still runs everything anyways. Seems like I might have to disable everything and then call EndTestCase. If I do this in the Setup phase, it will only mark the Setup phase as "Skipped". I have to run this as the first module in the Parent test case and have nothing in the Parent's Setup phase in order to get the desired result.
I am pretty sure I am doing this wrong.
That would give me a failure that I would then have to analyze to figure out that it's a false failure. Ideally, I only want to run the test case if the feature is enabled. I know I can write a module that can disable test cases that hasn't started yet by doing the following:
TestSuite.Current.GetTestCase(TESTCASENAME).checked = false;
But, that would force me to maintain a list of test case name. I was trying to be a little more dynamic. I was willing to compromise and get a "skipped" test reported. If I see that reported than I automatically know the reason.
Hi johnzer,
Calling EndTestCase(TestResult.Skipped) seems to just mark the test case as skip, but still runs everything anyways. Seems like I might have to disable everything and then call EndTestCase. If I do this in the Setup phase, it will only mark the Setup phase as "Skipped". I have to run this as the first module in the Parent test case and have nothing in the Parent's Setup phase in order to get the desired result.
I am pretty sure I am doing this wrong.
Re: How do I set the current test case status to "Skipped"
You should never have "false failures". If the test is setup correctly, then any failure should mean there is a bug in the SUT.
You should know exactly what the expected outcome is for every test case, so you should be able to code for it in each situation. Positive tests should be testing a positive SUT outcome and should pass if that happens. Negative tests should be testing a negative SUT outcome (error is thrown, etc.) and should pass if that happens.
If that is the case, then, again, there is no need to mark anything as skipped.
I would suggest that you may want to re-examine your testing methodology and make sure that you don't have any false negatives and that all testing has a known expected result given the test script and data combination.
You should know exactly what the expected outcome is for every test case, so you should be able to code for it in each situation. Positive tests should be testing a positive SUT outcome and should pass if that happens. Negative tests should be testing a negative SUT outcome (error is thrown, etc.) and should pass if that happens.
If that is the case, then, again, there is no need to mark anything as skipped.
I would suggest that you may want to re-examine your testing methodology and make sure that you don't have any false negatives and that all testing has a known expected result given the test script and data combination.
Shortcuts usually aren't...