Test Suite Error Behavior

Ask general questions here.
tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Test Suite Error Behavior

Post by tallahassee101 » Wed Mar 16, 2011 7:29 pm

Is there anyway to have the test suite DO an action on an error? I have code that I used before the test suite where if a test failed i would run a method that shutdown and restarted our application and put it in a "clean" mode. I would like to use this with the test suite, is that possible?

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Test Suite Error Behavior

Post by Ciege » Wed Mar 16, 2011 8:19 pm

I'm not using Test Suites, but according to the user guide it looks like you would need to set up the Test Case Setting's Error Behavior to "Continue With Sibling" and have an error check / cleanup sibling in between each Test Case.

If that is indeed the only way to control error cases, it seems like a good place for a feature request to allow the user to call a defined cleanup method if any Test Case results in an error.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Wed Mar 16, 2011 10:22 pm

Yea I saw the Continue with Sibling option, but sticking a sibling in between each set of tests seems like a bad implementation. If there is not another way to do this can you guys add this as a feature request because I would think this would be helpful to most users?

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Test Suite Error Behavior

Post by Support Team » Thu Mar 17, 2011 12:07 am

Hello,

When a validation fails an exception is thrown. You could convert the validation to user code via context menu on the action in the recording file and in the user code you can catch the exception and call your code or another recording that restores your application under test

Code: Select all

			try
			{
				Validate.Attribute(repo.FormVIP_Database.TextVIP_count__1Info, "Text", count);
			}
			catch
			{
				restoreModule.Start();
				throw;
			}
In the test suite you can mark a module as tear-down. This will be executed even if there was a failure. You could check in that module whether there was a failure and do something accordingly.


regards,
Roland
Ranorex Support Team
.
Image

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Thu Mar 17, 2011 2:25 pm

Thanks Peter. I didn't see that tear down option I will give that a try, it sounds like what we need, however it would be nice to have a feature option to run a module or user code only in the case of an error. In the case of using the tear down I would still need to put the tear-down test module in each of my test suites.

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Thu Mar 24, 2011 11:10 pm

So the problem I'm seeing is that in making a module a "tear down" it gets run if there is not an error as well. How can I find if an error HAS been thrown? I can't possibly change all my validation code to user code and put try/catches in there.
Basically what I want to do is if an error is thrown, restart our application and continue to the next test. Currently with the new test suite in 3.0 if an error is thrown in test 5 then every test after 5 will fail.

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Test Suite Error Behavior

Post by Support Team » Mon Mar 28, 2011 9:57 am

Hello,

here is an example teardown code that does what you want to do:
IReportItem Failed = null;
foreach (IReportItem rep in TestReport.CurrentTestCaseActivity.Children)
    if ((rep as Activity).Status == Ranorex.Core.Reporting.ActivityStatus.Failed)
        Failed = rep;

if (Failed != null)
{
    repo.FormVIP_Database.ButtonClose.Click();
    Host.Local.RunApplication("C:\\Ranorex\\samples\\VipApplication.exe", "");
}
Regards,
Roland
Ranorex Support Team
.
Image

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Wed Mar 30, 2011 4:17 pm

Roland,

Thanks for your help, but this code doesn't compile. Is this partly pseudo code or is there a library I need to include? IReportItem, TestReport, and Activity all do not exist in the context or could not be found when trying to compile.

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Test Suite Error Behavior

Post by Support Team » Wed Mar 30, 2011 10:23 pm

Hi,

you need to add using statements to your file:
using Ranorex;
using Ranorex.Core;
using Ranorex.Core.Testing;
using Ranorex.Core.Reporting;
Regards,
Roland
Ranorex Support Team
.
Image

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Thu Mar 31, 2011 7:05 pm

Thanks,

That builds now, however it does not work. I have a Test Case with two recording modules underneath it: CreateTrack and RestartSoftware (teardown method). Create Track has 7 iterations that run and it currently crashes on the 1st iteration and then the teardown method is run. The teardown code gets run but the following code never returns true:
if ((rep as Activity).Status == Ranorex.Core.Reporting.ActivityStatus.Failed)

I tried to take a look at the local variables with a break point, but Ranorex continuously freezes/crashes when I try to drill down into: TestReport.CurrentTestCaseActivity.Children fields.

Please help, right now we have to restart after each test case since we have no way of checking for errors in the test cases. This adds a lot of unnecessary time to our test runner.

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Test Suite Error Behavior

Post by Support Team » Thu Mar 31, 2011 9:12 pm

Is CreateTrack a test case with data source?

The code only is for the situation:

Code: Select all

TestCase
    Module1.rxrec or cs
    Module2.rxrec or cs
    ...
    teardown
       tdmodule.rxrec or cs
Could you outline your containment hierarchy or maybe attach a screenshot?

Regards,
Roland
Ranorex Support Team
.
Image

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Mon Apr 04, 2011 2:50 pm

My test case has a data source table, I'm not sure why that matters. I attached a screenshot of the section of our test case. RestartTacViewC2 is the name of the teardown module I want to run ONLY in the case of an error in the Ranorex test.
Attachments
RanorexProblem.png
RanorexProblem.png (20.78 KiB) Viewed 4142 times

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Tue Apr 05, 2011 7:48 pm

I did some more testing and it seems the code you provided only works if there are no data variables bound by the test case. Almost all of my test cases have bound variables, so how can I get this to work with these kinds of test cases?

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Test Suite Error Behavior

Post by Support Team » Wed Apr 06, 2011 2:52 pm

Hello,

this code is more flexible. It should also work for TestCases with Data Source.
//find failed module
        static class FailedFinder
        {
        	static public void Search(Activity act)
        	{
				static public Activity Failed = null;
				foreach (Activity child in act.Children)  
				{
					if (child is TestModuleActivity)
					{
							if (child.Status == Ranorex.Core.Reporting.ActivityStatus.Failed)
					       		Failed = child;  
					}
					else
						Search(child);
				}
        	}        	
        }

		//this is a user code method
        public void TestFail()
		{
			FailedFinder.Search(TestReport.CurrentTestCaseActivity);
			
			if (FailedFinder.Failed != null)  
			{  
				Report.Info("Failure in this test case");
			}

		}
Regards,
Roland
Ranorex Support Team
.
Image

tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Wed Apr 06, 2011 4:26 pm

Thank you Roland, that code does what I needed.