Repeating a test case upon failure

Ranorex Spy, Recorder, and Studio.
stevenh
Posts: 11
Joined: Wed Sep 05, 2012 9:51 pm

Repeating a test case upon failure

Post by stevenh » Thu Sep 27, 2012 5:45 pm

Hi,

I have been looking through the forums for a solution to repeat a test case after it has failed but was unable to come up with something concrete. After a test case has failed and it enters into it's teardown phase, I would like it to be able to restart the instance of the test case and run the test again. In order to do this, I figured there were two ways to accomplish this. Either simply make a call to the instance to run again or to setup a global parameter to control the amount of iterations that occur.

Any help is appreciated,
Steven

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Repeating a test case upon failure

Post by Support Team » Fri Sep 28, 2012 8:59 am

Hello Steven,

The easiest way to repeat the test case is to use Data-Driven Testing. You can set the error behavior setting to "Continue with iteration". The test case will repeat with the next iteration in case of an failure, but it will also repeat with the next iteration if no failure occurs. If you only want to start specific Recordings you can do so using the following code within the teardown region:
YourRecording.Start();
Regards,
Bernhard
Ranorex Support Team
.
Image

stevenh
Posts: 11
Joined: Wed Sep 05, 2012 9:51 pm

Re: Repeating a test case upon failure

Post by stevenh » Mon Oct 01, 2012 8:07 pm

Hi Bernhard,

I was looking more for a solution where I can repeat the current test case for all the test cases, which rules out the possibility of just calling a specific test case as I have a general teardown that is used for all the tests. If it was possible to iterate the same test case with the same set of variables, that would be great. Any more suggestions on this? Ranorex tests fail sometimes when trying to locate an element that was found before. Therefore, being able to repeat a test case from scratch will hopefully solve this problem.

Thanks,
Steven

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Repeating a test case upon failure

Post by Support Team » Tue Oct 02, 2012 12:37 pm

Hello Steven,

Does it help if you increase the search timeout to find the element instead of restarting the test case? Another thing you can try is to disable Folder Caching.

Regards,
Bernhard
Ranorex Support Team
.
Image

stevenh
Posts: 11
Joined: Wed Sep 05, 2012 9:51 pm

Re: Repeating a test case upon failure

Post by stevenh » Tue Oct 09, 2012 5:59 pm

Hi Bernhard,

Unfortunately, increasing the timeout length does not help. It is still preferred to repeat the test care upon failure. I noticed there are methods to run code modules and the test suite but I have not come across a method that will run a test case. I want to be able to run the same test case again so that in the report, it will appear as if it ran two iterations with the same parameters. Would this be possible to implement in the teardown?

Thanks,
Steven

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Repeating a test case upon failure

Post by Support Team » Wed Oct 10, 2012 5:19 pm

Hello Steven,

You could try to copy your test case and paste it after the original test case.
In the teardown region of your original test case you can check if the test case has failed. If this is the case you can activate the new test case as shown below. The specified test case will only be executed if the original test case has failed.
if (Ranorex.Core.Testing.TestCase.Current.Status.Equals(Ranorex.Core.Reporting.ActivityStatus.Failed))
{
	TestSuite.Current.GetTestCase("TestCase1").Checked = true;
}
else
{
	TestSuite.Current.GetTestCase("TestCase1").Checked = false;
}
I hope this workaround will help you!

Regards,
Bernhard
Ranorex Support Team
.
Image

stevenh
Posts: 11
Joined: Wed Sep 05, 2012 9:51 pm

Re: Repeating a test case upon failure

Post by stevenh » Wed Oct 10, 2012 9:28 pm

Hi Bernhard,

I was wondering if instead of doing that, is there a way to call a test case in code?

Thanks again,
Steven

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Repeating a test case upon failure

Post by Support Team » Fri Oct 12, 2012 12:05 pm

Hi Steven,

A test case is not designed to be executed from UserCode. There will be more possibilities in the test suite to work with conditions but meanwhile there is no other "supported" way.

Regards,
Markus
Ranorex Support Team
.
Image

stevenh
Posts: 11
Joined: Wed Sep 05, 2012 9:51 pm

Re: Repeating a test case upon failure

Post by stevenh » Fri Oct 12, 2012 9:13 pm

Hi Markus,

That is unfortunate. Can you go more into detail of the possibilities to work with conditions in the test suite? I appreciate the suggestion from Bernhard but it does not seem like a very "clean" way that I would prefer at this moment.

Thanks,
Steven

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Repeating a test case upon failure

Post by Support Team » Mon Oct 15, 2012 3:07 pm

Hello Steven,

The possibility to work with conditions is planned for one of our future releases. But it is not certain which version will provide this feature.

Regards,
Bernhard
Ranorex Support Team
.
Image

avinashp22
Posts: 2
Joined: Wed Jan 22, 2014 1:29 am

Re: Repeating a test case upon failure

Post by avinashp22 » Fri Apr 04, 2014 10:31 pm

Hi Ranorex Support,

To follow up on this thread, is there a way in Ranorex to re-try your test cases other than that what is mentioned above? Alternatively is there a way to add the test case again programmatically if the original test case has failed.
Also how will this work if it is a test suite setup you want to repeat instead of just the test case

Thanks,
Avinash

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: Repeating a test case upon failure

Post by krstcs » Mon Apr 07, 2014 2:10 pm

I would recommend that you look into using a Continuous Integration product (such as Jenkins, which is free) to do this.

Make your test cases very small, and create Jenkins jobs for each one. There is a plugin for Jenkins called "Naginator" that will allow you to specify how many times to rerun a job if it fails, and allows you to specify an interval to use in the repeats (so the second starts in 5 minutes after the first job, the third starts 10 minutes after the second, etc.).

This would probably be easier than trying to figure out how to do it in code. Plus, using a CI solution will allow you to do things in the future that will benefit you in the long run.
Shortcuts usually aren't...

macgowan
Posts: 14
Joined: Mon Jun 23, 2014 9:20 pm

Re: Repeating a test case upon failure

Post by macgowan » Thu Jun 26, 2014 6:35 pm

Hi ...

I'm responding to an old thread - so this questions may have been addressed. As discussed in the previous thread - I also am looking for a solution that will allow me to instantiate and run methods in other classes.

My use case was recommended by Ranorex Support for a workaround to a Flash/Flex bug. When testing against Flex objects - when the flex item is visible - Ranorex will fail to find the item and throw an exception (see below).

The workaround suggested by Ranorex support is to close the Flex Object and reopen and try again. This seems to work. The problem is intermittent.

Failed to find item 'SummaryRegressionRepository.JSA.LibraryReference.PrintButton'.
No element found for path '/dom[@domain~'[a-z, A-Z]*.workbench.judicial.int.westgroup.com']/body/div/div/div[@id~'[0-9]*_judicialsummary.trialCourtOrderSummaryPage.section.widget.libraryReferenceWindow']/../..//button[@automationname='LibraryReferenceWidget.printButton']' within 45.1s.

It seems data from the Flex Object is stored in the browser (IE.8) cache

We have created a WorkAround Test case that will execute before the suite of tests. The WorkAround Test will open all the flex objects prior to using the objects during the primary test.

Now the use case for access to Ranorex user classes ...

During the WorkAround Test a flex object will fail with a 'Failed to find item' exception. In these cases we would like to close the browser and begin again. We have several user classes for clearing data, closing a browser, opening a browser, setting data. It would be ideal if I could instantiate these classes and execute methods (.Run) from another class (i.e. WorkAround).

Below is some code.

The class below will clean data after a test process. It would be very cool if I could call this from another class in the Ranorex suite.

Code: Select all

// Removes all summaries for current artifact being used in test  
[TestModule("D8413E73-6B7F-4B63-9CCA-22C011D6402C", ModuleType.UserCode, 1)]
public class CleanupData : SummaryBase, ITestModule
{
    public CleanupData()
    { }

    void ITestModule.Run()
    {
        Report.Info("Removing all summaries in artifact with UUID = " + ArtifactUuid);
        SummaryRestClient client = new SummaryRestClient();
        client.deleteSummariesForArtifact(ArtifactUuid);
    }
}

The code below is a example of how it would be nice to call the class above
from the user class below. They are in different Code Modules.

See the IF Clause below

Code: Select all

/// <summary>
/// Description of WABAddNewHeadnote.
/// </summary>
[TestModule("CD0BBE37-4002-45EC-8B21-23A0FEDF4F3F", ModuleType.UserCode, 1)]
public class WABAddNewHeadnote : ITestModule
{

    SummaryRegressionRepository repo = SummaryRegressionRepository.Instance;

    /// <summary>
    /// Constructs a new instance.
    /// </summary>
    public WABAddNewHeadnote()
    {
        // Do not delete - a parameterless constructor is required!
    }

    /// <summary>
    /// This method will wait for the adapter to become visible. The Timer 
    /// object is created, configued and then started. We will wait until the 
    /// timer expires. If the timer will expire we will send a failure
    /// mesage to the family. 
    /// </summary>
    /// <remarks>You should not call this method directly, instead pass the module
    /// instance to the <see cref="TestModuleRunner.Run(ITestModule)"/> method
    /// that will in turn invoke this method.</remarks>
    void ITestModule.Run()
    {
        Mouse.DefaultMoveTime = 300;
        Keyboard.DefaultKeyPressTime = 100;
        Delay.SpeedFactor = 1.0;

        bool returnVal = true; 

        int retryCount = 0; 
        int retryLimit = 3;

        while (retryCount < retryLimit)
        { 
            Report.Info("Add new Headnote.");
            SummaryRegressionRepository.Instance.JSA.Features.CreateNewDropdown.Click();
            SummaryRegressionRepository.Instance.JSA.Features.AddNewHeadnote.Click();
            SummaryRegressionRepository.Instance.Alert.OKButton.Click();

            // Wait for the object to become visible 
            // waitForVisible() is not defined for this sample 

            if (!waitForVisible(SummaryRegressionRepository.Instance.JSA.Headnote.X))
            {
                // Headnote didn't open - will try again
                Report.Info("Fail: Headnote didn't open. RetryCount: ");

                // Here is where we need the ability to instantiate a user 
                // class and call methods. 
                // Something like below 

                CleanupData cleanupData = new CleanupData(); 
                cleanupData.Run(); 
            }
            else
            {
                Report.Info("Headnote opened successfully."); 
                break; 
            }
        } 
    }
}    



User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Repeating a test case upon failure

Post by Support Team » Wed Jul 02, 2014 3:38 pm

Hi,

I am afraid I didn't get the issue.
It is possible to start a code module from another module. You can even execute specific methods in the action table of a Recording, the Recording (in the Recording.UserCode.cs file) just needs to derive from the CodeModule where the method is defined in.
It is of course also possible to start specific code modules from code, you just need to use the TestModuleRunner.Run(ITestModule) method:
UserCodeModule1 userCodeModule = new UserCodeModule1(); // would be CleanupData in your case
TestModuleRunner.Run(userCodeModule);
Please let me know if I got this correctly.

Regards,
Markus
.
Image