Page 1 of 1

Long test runs intermittently fail with no indication why

Posted: Sat Feb 01, 2014 6:15 pm
by gcushman
I have a regression suite that consists of about 1100 testcases which take 25-26 hours to run using the TestSuite Runner. Ever since upgrading from 4.0.4 to 4.1.4, the suite intermittently fails (approx 10%). The failure is always at different spots and the report viewer doesn't display any information except for the following text:
"Data file (<filename>.rxlog.data) or transformation file is missing."

In Windows explorer I do see a tmp version of the file:
<filename>.rxlog.data_tmp

Examining the file, I can see that it just stopped writing to the file in mid-stream. I have plenty of free disk space on this box (10 gig free). I also checked event viewer around the time of the failure and no errors or warnings to indicate a problem. Also doesn't see any obvious signs of memory leaks and the overall health of the image when this happens appears to be fine.

Is this a known bug in 4.1?
Does Ranorex have a limitation with number of testcases per test suite? I'd prefer to not have to break them up into smaller test runs but if this keeps happening I will have to as it is frustrating to lose 24 hours. Searching through the partial log file for result="Failed" to determine failures combined with re-running the suite at the failure point is also an option but not a very user friendly one.

Any feedback greatly appreciated.
Thanks,
Greg
P.S. I attached screenshot of report viewer as well as zipped folder containing the partial report files.

Re: Long test runs intermittently fail with no indication why

Posted: Mon Feb 03, 2014 2:18 pm
by mebner
Hi,

I am not aware of such a bug.
Are you sure that nothing interrupts the test execution?
Do you disable the mouse and the keyboard in your test? If not please disable it since this will ensure that nobody can influence the tests by clicking the mouse or pressing a key. How you can disable it is described here: Disable Mouse/Keyboard Activities.

May I also ask you to enable the internal logging and to send me the created log file of a failed test run?
In order to enable the internal logging for your Ranorex test you just need to copy the attached NLog.config file to the same directory where your test exe resides.

Regards,
Markus

Re: Long test runs intermittently fail with no indication why

Posted: Mon Feb 03, 2014 9:29 pm
by gcushman
Thanks for the prompt reply. I am sure nobody is touching mouse/keypad and if something like that happened I would expect to see an error such as "unable to find <object>" error and even then it should proceed to the next testcase (the Error Behavior is set to continue to next iteration).

I will send you internal log file when I can capture one.

Thanks,
Greg

Re: Long test runs intermittently fail with no indication why

Posted: Wed May 06, 2015 4:13 pm
by gcushman
Great news, looks like this is fixed in 5.2.2. I've been running the same 1000+ testcases on an almost nightly basis for the last 2-3 months and have not seen this error since the upgrade.

Thanks Ranorex team!