Hi there
Since our update to version 7 we've found that certain tests seem to randomly fail.
The message we keep getting is "does not match the specified value (element is not valid)."
When we re-run the test it works fine but this may fail again on a later test execution.
When we rolled back to version 6 the issues stopped occurring.
We've since been adding delays immediately before the validations and we've found (although I can't fully confirm this) that the tests have started passing.
Any thoughts?
Thanks,
Peter
Tests randomly fail since update
Re: Tests randomly fail since update
I can now confirm that the delays are not working.
Re: Tests randomly fail since update
Hi,
I'm afraid, it's pretty hard to suggest something reliable without seeing the app in question, the test itself (at very least screenshot of recording where it typically fails) and exact error message (ideally entire report). But even then, it may be impossible to tell what's wrong.
I would suggest to try update Ranorex with most recent 7.1.1. If the problem still occurs, try to post as much details about the problem as possible. Also, if the issue happens at the same place, add action Report-Snapshot (before/after the failing place), which could help with problem identification.
BTW, if you are using hardcoded delays in your tests, you should definitely replace them with WaitForExists/NotExists actions. These are much more reliable in a long-term run than hardcoded delays.
I'm afraid, it's pretty hard to suggest something reliable without seeing the app in question, the test itself (at very least screenshot of recording where it typically fails) and exact error message (ideally entire report). But even then, it may be impossible to tell what's wrong.
I would suggest to try update Ranorex with most recent 7.1.1. If the problem still occurs, try to post as much details about the problem as possible. Also, if the issue happens at the same place, add action Report-Snapshot (before/after the failing place), which could help with problem identification.
BTW, if you are using hardcoded delays in your tests, you should definitely replace them with WaitForExists/NotExists actions. These are much more reliable in a long-term run than hardcoded delays.
Last edited by odklizec on Tue Aug 15, 2017 3:15 pm, edited 1 time in total.
Pavel Kudrys
Ranorex explorer at Descartes Systems
Please add these details to your questions:
Ranorex explorer at Descartes Systems
Please add these details to your questions:
- Ranorex Snapshot. Learn how to create one >here<
- Ranorex xPath of problematic element(s)
- Ranorex version
- OS version
- HW configuration
Re: Tests randomly fail since update
Hi thereodklizec wrote:Hi,
I'm afraid, it's pretty hard to suggest something reliable without seeing the app in question, the test itself (at very least screenshot of recording where it typically fails) and exact error message (ideally entire report). But even then, it may be impossible to tell what's wrong.
I would suggest to try update Ranorex with most recent 7.1.1. If the problem still occurs, try to post as much details about the problem a possible. Also, if the issue happens at the same place, add action Report-Snapshot (before/after the failing place), which could help with problem identification.
BTW, if you are using hardcoded delays in your step, you should definitely replace them with WaitForExists/NotExists actions. These are much more reliable in a long-term run than hardcoded delays.
I will update to the latest version and failing that I will post back with additional details
Thanks,
Peter
Re: Tests randomly fail since update
Hi againodklizec wrote:Hi,
I'm afraid, it's pretty hard to suggest something reliable without seeing the app in question, the test itself (at very least screenshot of recording where it typically fails) and exact error message (ideally entire report). But even then, it may be impossible to tell what's wrong.
I would suggest to try update Ranorex with most recent 7.1.1. If the problem still occurs, try to post as much details about the problem as possible. Also, if the issue happens at the same place, add action Report-Snapshot (before/after the failing place), which could help with problem identification.
BTW, if you are using hardcoded delays in your tests, you should definitely replace them with WaitForExists/NotExists actions. These are much more reliable in a long-term run than hardcoded delays.
I have updated to the latest version 7.1.1 and we're now getting more errors than before (there's a reference to winforms) It's not getting past the first few tests at all now. I've attached a zip file with the errors.
- Attachments
-
- Ranorex.zip
- (172.49 KiB) Downloaded 35 times
Re: Tests randomly fail since update
Have you looked at the release notes? You should read them every time you update to make sure you know the possible issues. In addition there were several API changes in 7.0.
https://www.ranorex.com/release-notes.html
https://www.ranorex.com/release-notes/api-changes.html
Specifically in 7.0:
https://www.ranorex.com/release-notes.html
https://www.ranorex.com/release-notes/api-changes.html
Specifically in 7.0:
UserCode methods expect a new alias for the System.Windows.Forms namespace. This can cause compile errors in user code classes after an upgrade complaining about "ambiguous references", e.g. between System.Windows.Forms.Control and Ranorex.Control
-> remove any existing "using System.Windows.Forms;" statement at the top of the file and prefix the ambiguous reference with either "WinForms." (most likely) or "Ranorex." depending on whether you want to reference a System.Windows.Forms or Ranorex type
Shortcuts usually aren't...
Re: Tests randomly fail since update
Hi therekrstcs wrote:Have you looked at the release notes? You should read them every time you update to make sure you know the possible issues. In addition there were several API changes in 7.0.
https://www.ranorex.com/release-notes.html
https://www.ranorex.com/release-notes/api-changes.html
Specifically in 7.0:
UserCode methods expect a new alias for the System.Windows.Forms namespace. This can cause compile errors in user code classes after an upgrade complaining about "ambiguous references", e.g. between System.Windows.Forms.Control and Ranorex.Control
-> remove any existing "using System.Windows.Forms;" statement at the top of the file and prefix the ambiguous reference with either "WinForms." (most likely) or "Ranorex." depending on whether you want to reference a System.Windows.Forms or Ranorex type
I've read through but I can't find anything related. The part you're referencing is also on a 6.2 release - not 7.
We've updated from 7.0.1 to 7.1.1
Re: Tests randomly fail since update
In addition - we're now receiving this message which could be the cause of our failures.
- Attachments
-
- Capture.PNG (40.64 KiB) Viewed 928 times
Re: Tests randomly fail since update
Yeah, you're right, I must have missed the transition on the page. Search failure on my part, apologies.
After looking at the text file you attached, it appears that the issue is that there is a problem with object recognition due to the use of .NET 1.1 version of WinForms, is that correct? What .NET framework does your AUT target?
Also, the report data file you attached is corrupted and can't be opened correctly. Please attach one that is complete.
It is possible that the LineageGlueRule exception is the problem, but they usually don't cause issues. They are thrown often on the back-end but don't usually cause tests to fail.
You might want to email support directly at [email protected] (if in North America) or [email protected].
After looking at the text file you attached, it appears that the issue is that there is a problem with object recognition due to the use of .NET 1.1 version of WinForms, is that correct? What .NET framework does your AUT target?
Also, the report data file you attached is corrupted and can't be opened correctly. Please attach one that is complete.
It is possible that the LineageGlueRule exception is the problem, but they usually don't cause issues. They are thrown often on the back-end but don't usually cause tests to fail.
You might want to email support directly at [email protected] (if in North America) or [email protected].
Shortcuts usually aren't...
Re: Tests randomly fail since update
Hi again
Yes I've just sent support an email this morning. Hopefully the problems can be rectified as our test suite is currently unusable. I will post the solutions when available
Yes I've just sent support an email this morning. Hopefully the problems can be rectified as our test suite is currently unusable. I will post the solutions when available
Re: Tests randomly fail since update
I've been in touch with the Ranorex support team. They've analysed our problem and have since raised our issue with their development team.
I will continue to post updates.
I will continue to post updates.