How to override Test case Status

Class library usage, coding and language questions.
sergii
Posts: 30
Joined: Fri Jun 07, 2013 11:07 pm

How to override Test case Status

Post by sergii » Tue Jun 24, 2014 1:42 am

I'm trying to make extra logic in the Final part of the test suite.

I'm analyzing if some of the Test cases failed (by iterating on TestSuite and checking is status == .Failed).
If some test case failed, I want to see if there was data in DB needed for it.

In case it was no data at that point of time - I want to mark Test Case as passed with Warning.


So I constructed dictionary<StringTCName, StringTCStatus> with all my test cases and statuses):

foreach (testcase in MyTestSuite.keys) {
if (MyTestSuite[testcase] == Ranorex.core.reporting.activitystatus.Failed) {
//call to DB to see if there was dat. If it was no data, I don't want to fail the whole suite because of false positive triggering
TestSuite.Current.gettestcase(textcase).status = Ranorex.core.reporting.activitystatus.Ignored
}
}


I know that there are workarounds as checking if there is data before executing TC, but for simplicity of the description of the whole problem, let's just assume that I cannot do that, and I need to analyze results of the test case execution. If TC failed for valid reason - mark it as ignored or success.

I can do manual check of status of all test cases and if all of them failed due to valid reasons - mark the whole test suite as success.

QUESTION: Is there a way to change result of the testcase run or the whole testsuite?

If there is no way I'm thinking about something like catching the failed test case exception and holding it for future. Then after all TCs finished - analyzing received exception (candidates for bugs) and checking with DB if it was data. But again, I need to update Status of testsuite programmatically.

krstcs
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: How to override Test case Status

Post by krstcs » Tue Jun 24, 2014 1:42 pm

I would very strongly discourage changing the results of a test. That type of thing should not be done.

If you need different results from a test based on data, then set the test up so that the data drives it correctly. For instance, if you want a test module returning a warning if there is no data passed to it then move all of the actions to user code and wrap them in an if-else block like this:

Code: Select all

if (data.Equals(""))
    {
        Report.Warn(...);
    } else {
        //do stuff here
    }
It is considered very bad practice to modify test results after the fact, and in some cases can be considered unethical, depending on what is changed.

If you want your tests to produce specific types of results, then code the tests so they produce those types of results every time they are run.
Shortcuts usually aren't...