Replay a test only on failure

[Chada]

Is it possible to set a condition to test to replay ONLY when it fails?

I know this is probably a long shot, but it is worth asking

For example, our test suite sometimes fail at random places because of environment activities (e.g. background processes that runs, etc). But when I kick them off a second time, they usually succeed. We are looking into different options to bypass that, but we were wondering if we can also somehow just replay a failed test automatically if we actually expect it to succeed. Repeating the whole test suite just to make sure it was one of those random environmental causes is quite time consuming, so this may save us quiite a bit of time.

Well, who knows, maybe I'm lucky

[Ciege]

Well my inclination is to say that your test code is not robust enough to handle these scenarios... If you get random failures you need to beef up your code to account for and eliminate these failures. Sorry...

But, if you are writing your own code (user code) you can trap some errors, send to a reset method to reset your AUT to a state where the test can be run again, and re-call the method that this particular test is in...

I think your time is better spent fixing the issues rather than applying a band-aid. IMHO...

[sdaly]

Totally agree with Ceige.... You need to go through a phase of continuously running your tests on different environments, during the day, during the night etc. Every time you get a false failure, you need to investigate why it failed and put more robust test code in place.

Some examples -

>No element exists for rxpath - Tweek rxpath, maybe it needs to be more flexible.
>Sometimes can't close app/browser properly in cleanup - Try close process if that fails then kill process
>Another process using file trying to edit - Try several more times before give up.
>Path failures with dates on single digit months but not double digit months (eg 8th, 28th) - make sure your paths search with the correct date format eg 08 instead of 8.
>Failures because of time - I have seen tests fail due to a component which only runs between a certain period of time, in this case, check the time and output a Report.Warn then fail the test if the time is outside the working range of the component.
>rxpaths different on different OS's - Try stay away from the native gui, for example, instead of going into service manager to restart a service, do it purely though code.
>Full text not being entered into input box before continuing - validate input tag value and re-enter if it is not as expected before continuing.
>Default data not available - run query against db to check for expected data and validate before continuing.
>rxpaths changing due to different views, eg extra columns being shown in a datagrid - dynamically find the column and use the column number in your path
>Different behaviour of your SUT on different OS's or device types (server/workstation) - add server/workstation detection and take required test path.

>Can't work out failure - add more report statements and screenshots.
>Add more validations