How to force a test iteration to skip

Class library usage, coding and language questions.
carsonw
Posts: 178
Joined: Tue Nov 08, 2011 10:01 pm

How to force a test iteration to skip

Post by carsonw » Thu May 10, 2012 7:41 pm

My test data from my datasheet may have hundreds of rows (so, hundreds of iterations). We pass variables into Ranorex to run specific rows, and ignore others (for exampled we might end up running rows 100-150, where rows 1-99 and 151-400 are not run).

However, the rows that are not run are showed as passing, and that makes it difficult to find which rows actually ran.

I can see how to End a Test case and test Module as "skipped" but I need to end an iteration as skipped as well.

In the screenshot attached you can see what I mean - see how all the subsequent test iterations are "skipped"? I want to be able to set a test iteration to that status myself. Thanks!
You do not have the required permissions to view the files attached to this post.

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: How to force a test iteration to skip

Post by Support Team » Mon May 14, 2012 1:25 pm

Hi,
However, the rows that are not run are showed as passing, and that makes it difficult to find which rows actually ran.
I am not sure I completely got your question. Do you mean that the iterations which colored green were not executed?
In the screenshot attached you can see what I mean - see how all the subsequent test iterations are "skipped"?
Maybe they are skipped because the error behavior was set to "Stop" instead of "Continue with iteration".
For more information please check the following link.

Regards,
Bernhard
Ranorex Support Team

carsonw
Posts: 178
Joined: Tue Nov 08, 2011 10:01 pm

Re: How to force a test iteration to skip

Post by carsonw » Mon May 14, 2012 8:39 pm

Support Team wrote:Hi,
Carson wrote:However, the rows that are not run are showed as passing, and that makes it difficult to find which rows actually ran.
I am not sure I completely got your question. Do you mean that the iterations which colored green were not executed?
I think this person is asking for the same thing, but in a different way: http://www.ranorex.com/forum/remove-war ... t3312.html
Essentially - the "passed" iterations technically ran, but they didn't do anything. We have a check to see if that particular iteration is "enabled", if it's not, it just does nothing and exits. I'm trying to have the report reflect this - so that in these cases where "nothing" happens, the test case shows as "skipped" or some other indication that I purposely skipped a test and it did not run. As it is now, my test has 400 iterations, but only a small subset usually runs... but I still see 400 passed tests so picking out the ones that actually did something is challenging.
Support Team wrote:
Carson wrote:In the screenshot attached you can see what I mean - see how all the subsequent test iterations are "skipped"?
Maybe they are skipped because the error behavior was set to "Stop" instead of "Continue with iteration".
For more information please check the following link.
It's ok that they were skipped, I was just showing you what I meant, not saying that it was a problem :)

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: How to force a test iteration to skip

Post by Support Team » Wed May 16, 2012 3:58 pm

carsonw wrote:I think this person is asking for the same thing, but in a different way: remove-warning-for-optional-action-t3312.html
Essentially - the "passed" iterations technically ran, but they didn't do anything. We have a check to see if that particular iteration is "enabled", if it's not, it just does nothing and exits. I'm trying to have the report reflect this - so that in these cases where "nothing" happens, the test case shows as "skipped" or some other indication that I purposely skipped a test and it did not run. As it is now, my test has 400 iterations, but only a small subset usually runs... but I still see 400 passed tests so picking out the ones that actually did something is challenging.
When I understand it correctly, you just want to abort the whole test suite if a test case fails? Means you don't want to see the blocked test cases in your report because the have no info for you?
FYI, we've added check-boxes where you can globally set filter inside the report to not show the blocked or success or failed test cases, means if have an open report you can select which test cases should be shown.

Regards,
Peter
Ranorex Team

carsonw
Posts: 178
Joined: Tue Nov 08, 2011 10:01 pm

Re: How to force a test iteration to skip

Post by carsonw » Wed May 16, 2012 5:31 pm

No not exactly, but I can see the word choice I made above was confusing - let me clarify...

I have one single test case, and that single test case is data bound to an excel sheet. And that sheet has 400 rows on it.

This single test case runs 400 times (400 iterations). However, depending on variables passed in to Ranorex, we may only run a subset of those 400 iteration (rows in the datasheet).

It's the same test case each time, but it's iterating through each of the rows. If it's a row that should not be executed, then we throw and catch an exception (saying test skipped or whatever).

On that iteration, nothing really happens, except to report that the test was skipped. The challenge, though, is that that iteration shows as passed. I want that iteration to show as skipped (not the entire case, just that row).

The reason is this - with 400 rows in the datasheet, I may only execute 60 of them in a particular test run. However, when I look at the results I see 400 passed cases even though 340 of them are just "test skipped". It makes it really hard to find the 60 I ran, because every single test is green / passed.

One workaround I've found, is reporting a warning on the skip - that way I know that any results with a warning symbol did not run. Then I can find the legitimate fails/passes easily.

However, I would prefer to use something other than "warning" because it looks like something went wrong. I was hoping to show that iteration as "skipped" (greyed out) so I know nothing really happened at all.

In the screenshot above, you can see that the subsequent iterations inside the test case show as skipped - so it's somehow possible to tag an iteration as skipped, I just can't see how to do it (unless it happens by virtue of skipping the rest of the test case, but I don't want to do that).

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: How to force a test iteration to skip

Post by Support Team » Fri May 18, 2012 4:46 pm

Hi,

thanks for the detailed explanation, now we got it :-).
We will ask our developers if there is a way to achieve this.

Regards,
Bernhard
Ranorex Support Team

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: How to force a test iteration to skip

Post by Support Team » Tue May 22, 2012 3:38 pm

Hi,

Here is the code which should make the trick.

This code snippet should be located in a Module of your TestCase:
if(iteration>yourLimit)
    ActivityStack.Current.Parent.CustomProperties["ignore"] = "";
You can also rename the property.
Don't forget to add "using Ranorex.Core.Reporting;" to the specific module.

This code snippet should be added to your Program.cs:
[STAThread]
        public static int Main(string[] args)
        {
            Keyboard.AbortKey = System.Windows.Forms.Keys.Pause;
            int error = 0;

            TestSuite.TestSuiteCompleted+= new EventHandler(TestSuite_TestSuiteCompleted);
            try
            {
                error = TestSuiteRunner.Run(typeof(Program), Environment.CommandLine);
            }
            catch (Exception e)
            {
                Report.Error("Unexpected exception occurred: " + e.ToString());
                error = -1;
            }
            return error;
        }

        static void TestSuite_TestSuiteCompleted(object sender, EventArgs e)
        {
          ActivityStack.Instance.VisitAll(a => {
                                            if (a.CustomProperties.ContainsKey("ignore"))
                                              a.Status = ActivityStatus.Ignored;
                                            return true;
                                          });
        }
This code will grey out the specific iterations in the report.
I hope this will help you solve the issue.

Regards,
Markus
Ranorex Support Team

kurts
Posts: 8
Joined: Thu Aug 02, 2012 9:09 pm

Re: How to force a test iteration to skip

Post by kurts » Thu Aug 02, 2012 9:14 pm

I've been trying to get this code to work and I'm still getting the steps I want skipped reported as successes.

The only thing I've done different is my code module is set up so if my excel sheet has a "Y" in one column it doesn't need to run that step. So my code is:

If (strskip =="N"){
//run the code
}else{
ActivityStack.Current.Parent.CustomProperties["ignore"] = "";
}

I've got the other code in my program.cs file as directed. Any ideas on how to get this work? I'm using version 3.3

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: How to force a test iteration to skip

Post by Support Team » Fri Aug 03, 2012 11:36 am

Hi,

I recently tested it with 3.3.1 and it worked as expected.
Can you execute the attached solution?

Regards,
Markus
Ranorex Support Team
You do not have the required permissions to view the files attached to this post.

kurts
Posts: 8
Joined: Thu Aug 02, 2012 9:09 pm

Re: How to force a test iteration to skip

Post by kurts » Fri Aug 03, 2012 2:11 pm

Your solution worked, but mine does not. My report does not look like yours. It doesn't list the iterations like yours but seems to count each one as a test run. I've attached a screen shot of it.

screen.jpg
You do not have the required permissions to view the files attached to this post.

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: How to force a test iteration to skip

Post by Support Team » Mon Aug 06, 2012 10:52 am

Hi,

Could it be that you have just executed your recording or that you are using a custom XSL file?
Are you using Ranorex Studio respectively a test suite?
It looks like you created your own test case hierarchy with VS.

Regards,
Markus
Ranorex Support Team

kurts
Posts: 8
Joined: Thu Aug 02, 2012 9:09 pm

Re: How to force a test iteration to skip

Post by kurts » Mon Aug 06, 2012 3:22 pm

I don't have a custom XLS file and I am running this in a test suite. I have one solution with several projects in that solution. What is strange is all of them output the reports that look like this. I added a new project today and the reports it produces look like the one you listed. Any ideas why the reports are different?

kurts
Posts: 8
Joined: Thu Aug 02, 2012 9:09 pm

Re: How to force a test iteration to skip

Post by kurts » Mon Aug 06, 2012 5:50 pm

I found my problem. The extension on the report file name was not correct. I am getting the reports correctly and suppressing the skipped steps.

Vaughan.Douglas
Posts: 254
Joined: Tue Mar 24, 2015 5:05 pm
Location: Des Moines, Iowa, USA

Re: How to force a test iteration to skip

Post by Vaughan.Douglas » Wed Sep 02, 2015 5:47 pm

Support Team wrote:Hi,

Here is the code which should make the trick.

This code snippet should be located in a Module of your TestCase:
if(iteration>yourLimit)
    ActivityStack.Current.Parent.CustomProperties["ignore"] = "";
You can also rename the property.
Don't forget to add "using Ranorex.Core.Reporting;" to the specific module.

This code snippet should be added to your Program.cs:
[STAThread]
        public static int Main(string[] args)
        {
            Keyboard.AbortKey = System.Windows.Forms.Keys.Pause;
            int error = 0;

            TestSuite.TestSuiteCompleted+= new EventHandler(TestSuite_TestSuiteCompleted);
            try
            {
                error = TestSuiteRunner.Run(typeof(Program), Environment.CommandLine);
            }
            catch (Exception e)
            {
                Report.Error("Unexpected exception occurred: " + e.ToString());
                error = -1;
            }
            return error;
        }

        static void TestSuite_TestSuiteCompleted(object sender, EventArgs e)
        {
          ActivityStack.Instance.VisitAll(a => {
                                            if (a.CustomProperties.ContainsKey("ignore"))
                                              a.Status = ActivityStatus.Ignored;
                                            return true;
                                          });
        }
This code will grey out the specific iterations in the report.
I hope this will help you solve the issue.

Regards,
Markus
Ranorex Support Team
Is there any way someone would be willing to provide this example in VB .Net, please?
Doug Vaughan

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 7470
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: How to force a test iteration to skip

Post by odklizec » Wed Sep 02, 2015 6:31 pm

Hi,

Ranorex has an ability to convert the C# code to VB .NET code and vice versa. Check this User Guide post...
http://www.ranorex.com/support/user-gui ... rsion.html
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration