SQL Data Sources incorrectly running via Open Test Solution

Bug reports.
RanoTester
Posts: 18
Joined: Mon Jan 14, 2013 7:16 pm

SQL Data Sources incorrectly running via Open Test Solution

Post by RanoTester » Thu Apr 28, 2016 6:08 pm

4_28_16 - Ranorex - Data Sources - A.JPG
4_28_16 - Ranorex - Data Sources - A.JPG (72.24 KiB) Viewed 790 times
PROBLEM SUMMARY: By launching Ranorex and opening our Test Solution, SQL driven Data Sources within that Solution are incorrectly, automatically running. This has been conclusively proven. The replication rate is 100%. This is occurring although those Data Sources have 'Auto-Load Data when Test Suite Opens' de-selected.

This appears to be totally incorrect and can create much havoc depending on what exactly this SQL does. This has resulted in numerous erroneous fails over the years and has cost our company a number of man hours troubleshooting failure type 'backend setting is changing on it's own'. This problem has also cost us time in replaying test cases that would have otherwise passed.

DESIRED BEHAVIOR: #1 is correct. #2 would seem correct.

1. SQL driven Data Sources (see attachment) should run at playback (if mapped to a given test case that is intended to play).

2. SQL driven Data Sources should automatically run when the Test Solution opens only if the data source has 'Auto-Load Data when Test Suite Opens' selected.

Thus far, our company is only using Ranorex - Data Sources so they can be mapped to Ranorex test cases and run at playback to create a desired playback condition. Thus far, this practice has been very successful. We're able to automate some complex scenarios that depend on very specific back-end conditions.

The only problem we have with this process is the above-stated issue of this SQL running when it should not. When we launch Ranorex and open a given Test Solution, we're not intending that simple action to actually execute / run the SQL in the various Data Sources. That would equate to a user in Microsoft Windows opening a folder containing a .exe file and the simple act of opening the folder actually running the .exe.

SYSTEM INFO:
Ranorex Studio Version 5.4.3.26106
Internet Explorer 11 Version 11.0.9600.18282 Update Version 11.0.20 (KB3058515)
Java Version 8 Update 65 (build 1.8.0_65-b17)
Adobe Flash Player - ActiveX Version: 18.0.0.261)
Windows 7 Ultimate N Service Pack 1 64-bit Operating System - Installed Memory: 4.00 GB - Processor - Intel(R) Core(TM @ 2.67GHz)

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: SQL Data Sources incorrectly running via Open Test Solution

Post by krstcs » Thu Apr 28, 2016 7:25 pm

Ranorex 5.4.3 is no longer supported.

Please update to Ranorex 5.4.6. If the problem continues, please report it.
Shortcuts usually aren't...

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: SQL Data Sources incorrectly running via Open Test Solution

Post by Support Team » Fri Apr 29, 2016 1:13 pm

Hi RanoTester,

Even if the problem you describe with the "Auto-Load Data..." option is somehow solved by Ranorex, you are on a very risky way:

The data connectors are not intended to change data (in particular the SQL connector), but to retrieve data. (For CSV data connectors, you can write back any changed data using the StoreData method, but this needs to be done in a test module, i.e. when the test is running.)
Using an "update" SQL query in a data connector will definitely cause problems.
For example, Ranorex will execute the query when you open the test case properties, see the variable binding, or just give you the number of iterations of a test case.
So, just looking at a test case or editing it may change your database.

IMHO a better test design would be:
Only get data using SQL Data Connectors, do not modify them. ("arrange")
In your test modules, update your SQL database and perform any actions. ("action")
At the end of your test case (last test module), assert that the right values are set. ("assert")

Regards,
Alex
Ranorex Team
.
Image

RanoTester
Posts: 18
Joined: Mon Jan 14, 2013 7:16 pm

Re: SQL Data Sources incorrectly running via Open Test Solution

Post by RanoTester » Fri Apr 29, 2016 10:01 pm

Thanks for the reply. We upgraded from Ranorex 5.4.3 to 5.4.6 and get the same behavior.

Please advise? I've reviewed the Ranorex User Guide and was unable to find out how to have Ranorex update the networked, Oracle database (used by our Web App) via an Action command in a Ranorex Recording Module.

The only way I see for Ranorex to control this database's setting at playback is via our current method . . . The one described in this ticket, i.e., having that SQL in a Data Source.
--> We then map that specific Data Source to a given test case that runs in a certain sequence so the playback tests our Web App with that backend setting in place. The front-end of our Web App can function quite differently depending on backend settings.

Additional Info:
1. We can live with the existing behavior (discussed in this ticket) because the workaround on our end is relatively easy.
--> Workaround Details: We keep our existing Ranorex setup and simply know what playback is occurring on all machines. If another machine is running the same Ranorex suite we would be opening, we observe what test case it is running. If it is one of the volatile ones (i.e., where Ranorex has had to update the backend setting in order for it to run), we'll know to not open Ranorex until that playback is complete.

2. The positives seem to out way the negatives. Our current method, i.e., Data Source, enables the playback to dictate very specific playback conditions on a network database, not a local database. Even if this SQL gets run unexpectedly (as discussed in this ticket), the only loss on our end (given the nature of this SQL) is lost man-hours / time to replay failed scripts that would have otherwise passed.
--> We don't intend on having SQL Updates (performed by Ranorex at playback) be more complex than toggling a given setting from "N" to "Y" or vice versa.

3. We control our test data at playback (when testing a number of our Web Apps) via a combination of static test data and C# User Code (i.e., where we have Ranorex analyze in real-time what data is appearing and act accordingly). Some of this C# is very advanced (Do - While loops, GetAttributeValueText, etc.).
--> We have the C# User Code update local and repository variable values on the fly. It enables the automaton to think on the spot and act according to what it finds at playback. The end result is Ranorex better able to handle dynamic, inconsistent playback conditions.
--> Some very complex scenarios have been tested this way. We're rely heavily on the C# User Code feature.

4. We essentially only use the SimpleDataConnector to provide a number of possible of Domains (so as to control how many times a given test case iterates at playback).

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: SQL Data Sources incorrectly running via Open Test Solution

Post by Support Team » Tue May 03, 2016 1:14 pm

Hi RanoTester,
Please advise? I've reviewed the Ranorex User Guide and was unable to find out how to have Ranorex update the networked, Oracle database (used by our Web App) via an Action command in a Ranorex Recording Module.
I think my colleague meant to update the database in the module using the .NET Framework. I'm afraid you'll have to write code in order to do so.
Additionally, the Ranorex SQL data connector was made only for receiving data from a database.
If you want to discuss your scenario on more detail, we can have a remote session. Please send an email to [email protected].

Regards,
Bernhard
.
Image