Test Suite Error Behavior

Ask general questions here.
tallahassee101
Posts: 169
Joined: Thu Jan 13, 2011 2:06 pm

Re: Test Suite Error Behavior

Post by tallahassee101 » Thu Apr 07, 2011 2:31 pm

Roland,

Actually this code is not working properly either. My test case hierarchy is as follows:

Code: Select all

Master Test Suite
  Parent Test
     TestA
         Test1
         Test2
         Test3
         teardownA <user code module containing your code>
     TestB
         Test1
         Test2
         Test3
         teardownB <user code module containing your code>
If Test 1,2, or 3 fail in TestA then teardownA will run (as I would expect), however this has the unwanted affect of making TestA, Parent Test, and Master Test Suite's states "Failure". When TestB is run after TestA and Tests 1-3 pass, teardownB is run (even though TestB test case is actually a success).

This upwards failing effect bug I noted in the following post: http://www.ranorex.com/forum/turning-of ... t2138.html

But the issue I have remains regardless as I want to know if any of the Tests in the CURRENT Test Case ( ie. TestA or TestB ) failed regardless of the state of the Parent Test. Is this possible?

Pixi6s
Posts: 92
Joined: Tue Jun 28, 2011 8:57 pm

Re: Test Suite Error Behavior

Post by Pixi6s » Mon Sep 26, 2011 9:58 pm

Support Team wrote:
//find failed module
        static class FailedFinder
        {
        	static public void Search(Activity act)
        	{
				static public Activity Failed = null;
				foreach (Activity child in act.Children)  
				{
					if (child is TestModuleActivity)
					{
							if (child.Status == Ranorex.Core.Reporting.ActivityStatus.Failed)
					       		Failed = child;  
					}
					else
						Search(child);
				}
        	}        	
        }

		//this is a user code method
        public void TestFail()
		{
			FailedFinder.Search(TestReport.CurrentTestCaseActivity);
			
			if (FailedFinder.Failed != null)  
			{  
				Report.Info("Failure in this test case");
			}

		}
I think Roland's last code snippit is what I am looking for. I have a nested testcase that call several tests. The nested testcase iterates several times depending on rows returned from a database query.

If one of the tests fails on something random within the nested testcase, I would like that iteration to do a cleanup action before continuing onto the next iteration - or that next iteration may fail because the website is not in the expected state.

I am using Ranorex 3.0.3 and C#. I see that TestFail calls FailedFinder, but where do I put these 2 method within my user code module? Can you give me a screen shot or attach a file of what the user code file would look like?

thanks
Sierra

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Test Suite Error Behavior

Post by Support Team » Wed Sep 28, 2011 3:26 pm

Hi,

You can use Roland's code or this one:
ITestCase iCase = TestSuite.Current.GetTestCase("TestCase1"); // The name of your Test Case
      if(iCase.Status == Ranorex.Core.Reporting.ActivityStatus.Failed){
        Report.Info("TestCase1 Failed!");
      }
you have to add this for instance into a method in your UserCode file from your TearDown folder.
The same holds for Roland's "TestFail" method.
But keep in mind that if you check the state of your actual Test Case in the TearDown of your actual Test Case you will get back "Active" and not "Failed", because the actual Test Case is still running.
As workaround you can add a new Test Case after the specific one and check the state of the "older" Test Case or you can check the state of the older Test Case in the setup of the actual one.
This is a known issue and we will fix it in a future release.

Regards,
Markus
Ranorex Support Team
.
Image

Pixi6s
Posts: 92
Joined: Tue Jun 28, 2011 8:57 pm

Re: Test Suite Error Behavior

Post by Pixi6s » Wed Sep 28, 2011 4:51 pm

Thanks for the information. This looks very similar to a previous code snippit given (I believe also by Roland) that was determined doesn't work with data sources. This is why I skipped passed it.

Will the code you provided work with data sources?

Where do I add this? Does it go in the

Code: Select all

void ITestModule.Run()
module?

Pixi6s
Posts: 92
Joined: Tue Jun 28, 2011 8:57 pm

Re: Test Suite Error Behavior

Post by Pixi6s » Wed Sep 28, 2011 7:13 pm

Well I tried doing it anyway and added it to void ITestModule.Run() and it I get 2 errors about assembly references. It doesn't seem to like GetTestCase is association with TestSuite. Are you sure this is a correct code snippet or is there something additional I need to add? My code looks like your except it has a real testcase name in it.

'Ranorex.Core.Testing.TestSuite' does not contain a definition for 'GetTestCase' and no extension method 'GetTestCase' accepting a first argument of type 'Ranorex.Core.Testing.TestSuite' could be found (are you missing a using directive or an assembly reference?) (CS1061)

The type or namespace name 'ITestCase' could not be found (are you missing a using directive or an assembly reference?) (CS0246)

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Test Suite Error Behavior

Post by Ciege » Wed Sep 28, 2011 7:54 pm

Do you have a reference to Ranorex.Core in your solution?

What version of Ranorex are you using? If you do have the proper reference you may need to upgrade your version of Ranorex.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

Pixi6s
Posts: 92
Joined: Tue Jun 28, 2011 8:57 pm

Re: Test Suite Error Behavior

Post by Pixi6s » Wed Sep 28, 2011 8:07 pm

I do have Ranorex.Core. I am on 3.03 using C#. I can upgrade to 3.1.

Pixi6s
Posts: 92
Joined: Tue Jun 28, 2011 8:57 pm

Re: Test Suite Error Behavior

Post by Pixi6s » Wed Sep 28, 2011 9:31 pm

Error are gone in 3.1, thanks

Pixi6s
Posts: 92
Joined: Tue Jun 28, 2011 8:57 pm

Re: Test Suite Error Behavior

Post by Pixi6s » Wed Sep 28, 2011 10:16 pm

Support Team wrote: But keep in mind that if you check the state of your actual Test Case in the TearDown of your actual Test Case you will get back "Active" and not "Failed", because the actual Test Case is still running.
As workaround you can add a new Test Case after the specific one and check the state of the "older" Test Case or you can check the state of the older Test Case in the setup of the actual one.
This is a known issue and we will fix it in a future release.

Regards,
Markus
Ranorex Support Team
I just actually understood this, and it's really messy. I also thought, for some reason, that the teardown was only going to get called if I have a failure. I guess that it's the case, it's more that the code within the teardown will only get called.

not horrible but not great. I will keep searching.

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Test Suite Error Behavior

Post by Support Team » Thu Sep 29, 2011 1:37 pm

Hi,
I just actually understood this, and it's really messy. I also thought, for some reason, that the teardown was only going to get called if I have a failure.
We know that this is not the cleanest or best solution, but at the moment there is no other way and no the teardown is always called.
As written we are working on this topic and will improve it in a next release.

Regards,
Markus
Ranorex Support Team
.
Image

Pixi6s
Posts: 92
Joined: Tue Jun 28, 2011 8:57 pm

Re: Test Suite Error Behavior

Post by Pixi6s » Fri Sep 30, 2011 6:20 pm

My clean-up step for a failure and for the end of an iteration happens to be the same in this case so I created a simple teardown method to do that step in all cases.

thanks