Screenshots - Group Test Steps - Image Comparison Questions : General Questions

[aunal]

Hello,

We're still in the evaluation process and here are a couple of questions that I have encountered today.

Ranorex takes screenshots of related components while recording a test case, which is, imo, very useful for the tester. However, once a portion of a test case is prepared, adding components from Spy to the Repository and Copy-Pasting actions by changing only the components lets us prepare the test cases faster, but previously added screenshots are copied along with the copied steps and new screenshots are NOT taken when the test case is executed. This causes incorrect screenshots related with the steps, thus confusing the test developer.

Is there a way to tell Ranorex to re-screenshoot while executing the test?

In addition to that, is there a way to group a severel steps that does the same thing and re-use it in different test cases? I know, we can do it by copy pasting but wouldn't it be nice to add only a step, let's say named "Close Active Panel", that includes the steps that needs to be done in order to close a panel.

Ranorex displays failed validation steps in detail in the prepared report. I mean, it say I compared "X" with "Y" and failed. However, it does not create a detailed report for passed steps. Is there a way to change the way reports are created or do we have to code our own report generation script?

What does 100% comparison mean in image comparison? I expect the two screenshots be exactly the same when 100% is selected. However, similar but different screenshots appear to have been passed even if 100% is selected.

Thank you in advance.

My configuration is as follows:
Ranorex: 3.2.0.15301
.Net RunTime Version: 2.0.50727.3625
Java: 1.6.0-b105
O/S: Windows XP with SP3

Best Regards,
Arda Unal

[Support Team]

Hi,

Is there a way to tell Ranorex to re-screenshoot while executing the test?

Sorry but such functionality is currenlty not available, but I will add a feature request for it.

n addition to that, is there a way to group a severel steps that does the same thing and re-use it in different test cases? I know, we can do it by copy pasting but wouldn't it be nice to add only a step, let's say named "Close Active Panel", that includes the steps that needs to be done in order to close a panel.

With Ranorex 3.3 we will add Module Groups. Module Groups are elements where you can create routines and add it to your test suite as a group.

Ranorex displays failed validation steps in detail in the prepared report. I mean, it say I compared "X" with "Y" and failed. However, it does not create a detailed report for passed steps. Is there a way to change the way reports are created or do we have to code our own report generation script?

You can create your own Report if you want, therefore please take a look to following blog http://www.ranorex.com/blog/customizing-ranorex-reports

What does 100% comparison mean in image comparison? I expect the two screenshots be exactly the same when 100% is selected. However, similar but different screenshots appear to have been passed even if 100% is selected.

Could it be that you have selected a region from the image which you compare and this region never changes? It depends on the region of the image you try to compare. If you select the whole image it fails too?

Regards,
Peter
Ranorex Team

[anja]

Hi,

the Option for grouping modules seems to be very interesting.... Is there already a Release Date planned for Ranorex 3.3?

Thank you...

[aunal]

Could it be that you have selected a region from the image which you compare and this region never changes? It depends on the region of the image you try to compare. If you select the whole image it fails too?

Well, actually, I didn't select any area, I chose to take a screenshot of a component, which in this case a JPanel inlcuding a bitmap image. What I did is, I selected ImageComparison for the validation step, selected previously added screenshot to compare with one taken during test execution.

The result didn't change when I chose a region that is entirely changing.

Now that, I have gone over the generated code, I found out that;

Imaging.FindOptions is set to the following parameters: (0.9;None;337,213,811,668;True;10000000;0ms)

It seems that once you have added a ImageComparison validation step, changes made into Image Comparison options is not reflected in the generated code. I mean, if I change Image Comparison Similarity value after adding the validation step, generated code is not updated to reflect the change in Comparison Similarity value. It is, probably, applicable to other Imaging.FindOptions parameters as well.

Also, what does 10000000 (DownsizeTargetCost) do? What would happen if I change it to 1 for instance?

I have another question about validation. I would appreciate if you could answer that as well, even if it is not in the scope of this topic.

I'm trying to compare a value stored in a JTextField. Field stores a Double value but display only the Integer (actually it is rounded to the next Intgere value) part of it. Double value is truncated and displayed as truncated. A stored value like 235.33467 is displayed as 235 or 456.7635 is displayed as 457 (note that it is rounded to closest integer). What I want to compare is the displayed value not the stored one. Is there way to tell Ranorex to do such things? In addition, I couldn't get enough info about Regular Expressions in the User Guide. Are there any examples about number comparisons that you can direct me?

Thank you in advance.

Best Regards,
Arda Unal

[Support Team]

Hi,

For a better understanding how the image based automation works in Ranorex, please take a look to following documentation
http://www.ranorex.com/support/user-gui ... ation.html

Also, what does 10000000 (DownsizeTargetCost) do? What would happen if I change it to 1 for instance?

Please take a look to following API Doc
http://www.ranorex.com/Documentation/Ra ... tCosts.htm

I'm trying to compare a value stored in a JTextField. Field stores a Double value but display only the Integer (actually it is rounded to the next Intgere value) part of it. Double value is truncated and displayed as truncated. A stored value like 235.33467 is displayed as 235 or 456.7635 is displayed as 457 (note that it is rounded to closest integer). What I want to compare is the displayed value not the stored one. Is there way to tell Ranorex to do such things? In addition, I couldn't get enough info about Regular Expressions in the User Guide. Are there any examples about number comparisons that you can direct me?

Normally Ranorex converts the TextValue of the JTextField to a string. Did you try the following code?

Text someText = "yourInputPath";
var doubeValue = someText.TextValue;

I tried this on following sample app and it works as expected.
http://docs.oracle.com/javase/tutorial/ ... field.html
With Ranorex 3.3 the Java Plugin will support floating values too which is currently not available.

Regards,
Peter
Ranorex Team

[aunal]

So, Can we say that "Comparison with Margins" will also be available with Ranorex 3.3 or will we have to use User Code for that as well?

Thanks in advance.

Best Regards,
Arda Unal

[Support Team]

So, Can we say that "Comparison with Margins" will also be available with Ranorex 3.3 or will we have to use User Code for that as well?

Sorry but I have absolutely no clue what you mean with "Comparison with Margins". Please could you explain this in more detail?

Regards,
Peter
Ranorex Team

[aunal]

sure, here it is:

You have a number that you want to compare against a previously calculated value, probably an output of an algorithm. As expected, there are some round up errors during the calculation process or some estimations maybe, which you have to consider, so you know that the result is not exact value but it is in within the error limits of a calculated value.

Assume that you have the following formula, which you want to test: C = 3πr, 1 < r < 3

which makes;

C = 6π ~ 18.85

It is actually something like 18.849555....

So, what you need to do is; to define a margin around the expected value, which can be 18.8 ± 0.1 in this specific case. Note that ± 0.1 can change according to your desired precision, AUT, etc.

[Support Team]

Hi,

Now I got it. For sure this will be possible in Ranorex 3.3 if you use a User Code Module.

Regards,
Peter
Ranorex Team

[aunal]

Well, what about Ranorex 3.4 then?

The reason that I'm asking; this is really a big issue for us and I don't want to use User Code Module for almost every field in my AUT.

[Ciege]

With "user code" you only need to write a single method then pass the proper parameters to it (such as the object you are interacting with) to get the results you are after. Write it once, use it several times...

[aunal]

I'm well aware of that but not every tester have the necessary development skills, plus this is one of the basics of number comparison. I'm in the process of evaluating this product and I don't want to come up with a solution that can only be used by a few testers.

[Ciege]

In your test automation team do you not have a "framework guy" who is capable to write such extension methods and then lower level automation testers that run the tests and can perform simple record/playback? I've been in the test automation field for 15 years at several companies. I have also contracted with companies to help get their test automation team established. I have always taught that test automation is a profession that requires a certain amount of skill if you want to succeed. Thus, having a team that is capable at extending the built in record/playback functionality of any test automation software is paramount. That being said, I think you would be hard pressed to find any test automation package that will do *everything* you would ever want to do without a little programming skills or at least a central member on your team that is capable to be the framework guru.
Test automation is an expensive process (software cost as well as personnel) that if done properly will pay dividends. But if you set up with a team that is not capable, will be a costly mistake.

I stand by my post that writing a single method, in a framework DLL, that can be referenced by any of your testers would be the way to go.

[aunal]

Thank you for your input. but I think you have missed my point, "Comparison with Margins" is not something that is too eccentric, this is a basic and crucial point in number comparison. If I'm going to outsource my automation framework, I expect the application to at least do the basics for me.

Anyway, I will wait for an answer from one of the Support guys over here. Thanks.

[Support Team]

Hi,

Thank you for your input. but I think you have missed my point, "Comparison with Margins" is not something that is too eccentric, this is a basic and crucial point in number comparison. If I'm going to outsource my automation framework, I expect the application to at least do the basics for me.
Anyway, I will wait for an answer from one of the Support guys over here. Thanks.

A value range validation and complex validation is planned for this year. But I cannot tell you an exact date when this features will be available. Complex validation means, that you are able to compare more than one elements at one time.
Meanwhile you have to use the code based solution as ciege suggested already.

Regards,
Peter
Ranorex Team