Test Suite Error Behavior
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Test Suite Error Behavior
Is there anyway to have the test suite DO an action on an error? I have code that I used before the test suite where if a test failed i would run a method that shutdown and restarted our application and put it in a "clean" mode. I would like to use this with the test suite, is that possible?
Re: Test Suite Error Behavior
I'm not using Test Suites, but according to the user guide it looks like you would need to set up the Test Case Setting's Error Behavior to "Continue With Sibling" and have an error check / cleanup sibling in between each Test Case.
If that is indeed the only way to control error cases, it seems like a good place for a feature request to allow the user to call a defined cleanup method if any Test Case results in an error.
If that is indeed the only way to control error cases, it seems like a good place for a feature request to allow the user to call a defined cleanup method if any Test Case results in an error.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!
Ciege...
Ciege...
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
Yea I saw the Continue with Sibling option, but sticking a sibling in between each set of tests seems like a bad implementation. If there is not another way to do this can you guys add this as a feature request because I would think this would be helpful to most users?
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Test Suite Error Behavior
Hello,
When a validation fails an exception is thrown. You could convert the validation to user code via context menu on the action in the recording file and in the user code you can catch the exception and call your code or another recording that restores your application under test
In the test suite you can mark a module as tear-down. This will be executed even if there was a failure. You could check in that module whether there was a failure and do something accordingly.
regards,
Roland
Ranorex Support Team
When a validation fails an exception is thrown. You could convert the validation to user code via context menu on the action in the recording file and in the user code you can catch the exception and call your code or another recording that restores your application under test
Code: Select all
try
{
Validate.Attribute(repo.FormVIP_Database.TextVIP_count__1Info, "Text", count);
}
catch
{
restoreModule.Start();
throw;
}
regards,
Roland
Ranorex Support Team
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
Thanks Peter. I didn't see that tear down option I will give that a try, it sounds like what we need, however it would be nice to have a feature option to run a module or user code only in the case of an error. In the case of using the tear down I would still need to put the tear-down test module in each of my test suites.
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
So the problem I'm seeing is that in making a module a "tear down" it gets run if there is not an error as well. How can I find if an error HAS been thrown? I can't possibly change all my validation code to user code and put try/catches in there.
Basically what I want to do is if an error is thrown, restart our application and continue to the next test. Currently with the new test suite in 3.0 if an error is thrown in test 5 then every test after 5 will fail.
Basically what I want to do is if an error is thrown, restart our application and continue to the next test. Currently with the new test suite in 3.0 if an error is thrown in test 5 then every test after 5 will fail.
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Test Suite Error Behavior
Hello,
here is an example teardown code that does what you want to do:
Roland
Ranorex Support Team
here is an example teardown code that does what you want to do:
IReportItem Failed = null; foreach (IReportItem rep in TestReport.CurrentTestCaseActivity.Children) if ((rep as Activity).Status == Ranorex.Core.Reporting.ActivityStatus.Failed) Failed = rep; if (Failed != null) { repo.FormVIP_Database.ButtonClose.Click(); Host.Local.RunApplication("C:\\Ranorex\\samples\\VipApplication.exe", ""); }Regards,
Roland
Ranorex Support Team
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
Roland,
Thanks for your help, but this code doesn't compile. Is this partly pseudo code or is there a library I need to include? IReportItem, TestReport, and Activity all do not exist in the context or could not be found when trying to compile.
Thanks for your help, but this code doesn't compile. Is this partly pseudo code or is there a library I need to include? IReportItem, TestReport, and Activity all do not exist in the context or could not be found when trying to compile.
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Test Suite Error Behavior
Hi,
you need to add using statements to your file:
Roland
Ranorex Support Team
you need to add using statements to your file:
using Ranorex; using Ranorex.Core; using Ranorex.Core.Testing; using Ranorex.Core.Reporting;Regards,
Roland
Ranorex Support Team
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
Thanks,
That builds now, however it does not work. I have a Test Case with two recording modules underneath it: CreateTrack and RestartSoftware (teardown method). Create Track has 7 iterations that run and it currently crashes on the 1st iteration and then the teardown method is run. The teardown code gets run but the following code never returns true:
if ((rep as Activity).Status == Ranorex.Core.Reporting.ActivityStatus.Failed)
I tried to take a look at the local variables with a break point, but Ranorex continuously freezes/crashes when I try to drill down into: TestReport.CurrentTestCaseActivity.Children fields.
Please help, right now we have to restart after each test case since we have no way of checking for errors in the test cases. This adds a lot of unnecessary time to our test runner.
That builds now, however it does not work. I have a Test Case with two recording modules underneath it: CreateTrack and RestartSoftware (teardown method). Create Track has 7 iterations that run and it currently crashes on the 1st iteration and then the teardown method is run. The teardown code gets run but the following code never returns true:
if ((rep as Activity).Status == Ranorex.Core.Reporting.ActivityStatus.Failed)
I tried to take a look at the local variables with a break point, but Ranorex continuously freezes/crashes when I try to drill down into: TestReport.CurrentTestCaseActivity.Children fields.
Please help, right now we have to restart after each test case since we have no way of checking for errors in the test cases. This adds a lot of unnecessary time to our test runner.
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Test Suite Error Behavior
Is CreateTrack a test case with data source?
The code only is for the situation:
Could you outline your containment hierarchy or maybe attach a screenshot?
Regards,
Roland
Ranorex Support Team
The code only is for the situation:
Code: Select all
TestCase
Module1.rxrec or cs
Module2.rxrec or cs
...
teardown
tdmodule.rxrec or cs
Regards,
Roland
Ranorex Support Team
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
My test case has a data source table, I'm not sure why that matters. I attached a screenshot of the section of our test case. RestartTacViewC2 is the name of the teardown module I want to run ONLY in the case of an error in the Ranorex test.
You do not have the required permissions to view the files attached to this post.
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
I did some more testing and it seems the code you provided only works if there are no data variables bound by the test case. Almost all of my test cases have bound variables, so how can I get this to work with these kinds of test cases?
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Test Suite Error Behavior
Hello,
this code is more flexible. It should also work for TestCases with Data Source.
Roland
Ranorex Support Team
this code is more flexible. It should also work for TestCases with Data Source.
//find failed module static class FailedFinder { static public void Search(Activity act) { static public Activity Failed = null; foreach (Activity child in act.Children) { if (child is TestModuleActivity) { if (child.Status == Ranorex.Core.Reporting.ActivityStatus.Failed) Failed = child; } else Search(child); } } } //this is a user code method public void TestFail() { FailedFinder.Search(TestReport.CurrentTestCaseActivity); if (FailedFinder.Failed != null) { Report.Info("Failure in this test case"); } }Regards,
Roland
Ranorex Support Team
-
- Posts: 169
- Joined: Thu Jan 13, 2011 2:06 pm
Re: Test Suite Error Behavior
Thank you Roland, that code does what I needed.