whitch way is the best to integrate many reports?

Ask general questions here.
marcushe
Posts: 112
Joined: Tue Apr 14, 2009 6:38 am

whitch way is the best to integrate many reports?

Post by marcushe » Tue Jun 09, 2009 3:20 am

Environment: Win7 RC US/Muti-languages, Ranorex2.1

First,thanks for your Rnorex2.1.
We'll develop 20~50 test cases on 21 languages. I'm considering how to integrate so many reports . 20~50 test cases will create 20~50 reports.
1. Should i pour all info into 1 report , report.setup in next recording file will destory the previous content inputed?
2. Or should i combine all projects into 1 huge project ?
3. Or Is there some simple ways to integrate these reports?
Tester's job is execute test cases and read reports , after running so many cases , to let them know how many cases fail aggregately on 1 report ,which contain all reports as a treeview, and unfold each section to review result , is a good way.

Can you give some advices ,thanks?

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Post by Ciege » Tue Jun 09, 2009 4:11 pm

These are just some thoughts, take them for what they are worth...

1st, When you use Report.Setup for each of your tests I would suggest having a mechanism to automatically name your reports differently. Like TestName_time_date stamped so that each one is different. Try keeping them all in the same directory (for the next suggestion).

2nd, since all the Ranorex reports are XML, you can write your own HTML page that you can drop in that same directory (like an index). Have this index automatically create a link for each XML report in your test report directory.

3rd, you can take it one step further. In your index HTML have it read each XML file and look for a failure. If it find one have your Index create the XML like with a red FAIL next to the link. If it doesn't find a failure, have you index create a green PASS next to the XML link.

4th, if possible, have your Reports stored on a networked machine (not the test machine(s)) in a standardized directory structure. This way, anyone up the food chain with a web browser (i.e. your boss) can look at the reports results whenever they want to. Bosses love to think they have control over how they get and receive their status updates!

Example Index.HTML:

Reports in Test Directory
Test1_6_9_2009__10_01_AM PASS
Test2_6_9_2009__10_10_AM PASS
Test3_6_9_2009__11_15_AM FAIL
Test4_6_9_2009__11_20_AM PASS

marcushe
Posts: 112
Joined: Tue Apr 14, 2009 6:38 am

Thanks

Post by marcushe » Wed Jun 10, 2009 1:55 am

Thanks ,Ciege

Glad to get this idea.

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Post by Ciege » Wed Jun 10, 2009 4:53 pm

Glad I could help with some ideas. I just ask that when you make your final decisions and create something that you share it (or at least the ideas) with the group so we can all benefit from it.
You know helping each other out and help strengthen the Ranorex community!

devenb
Posts: 24
Joined: Mon Mar 09, 2009 11:50 am
Location: Bracknell, UK

Re: whitch way is the best to integrate many reports?

Post by devenb » Fri Jun 19, 2009 4:36 pm

Hi Ciege, Marcushe,

That was a great way to explain Ciege.
Even I faced a problem with the Reporting feature in Ranorex. The report which gets generated after execution cannot be mailed to the business stakeholders, since Ranorex.ReportViewer.exe sits on the test server where Ranorex is installed and I think it is needed to view the "*.rxlog" reports.
What I did was to create a batch file which would dig into the *.rxlog report and then create a testreport.html containing details like TestCaseNumber, TestCaseType - Validation or User, TestCaseResult and TestCaseStatus. The failed testcases or validations are marked in Red. The same report then can be mailed across to business stakeholders as an attachment. The batch file is still in its infancy and I would be adding summary of the test execution to it which would detail number of testcases executed, testcases passed, failed etc.
Hope they add a feature in upcoming version where the test report can be mailed across as an attachment.
If anyone needs any help with it I am happy to share what I have done.

Regards,
Deven Bairat

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: whitch way is the best to integrate many reports?

Post by Ciege » Fri Jun 19, 2009 4:45 pm

Deven, you do realize that if you just rename the .rxlog report file to .XML you can open the report no problem with Internet Explorer? You can then email the report.XML and the corresponding stylesheet to anyone to view. You do not need to use the extra batch file you crated to extract that data.

I took that one step further. When I use Report.Setup I actually name the report .XML so I skip the entire renaming step all together.

Code: Select all

string LogFile = @"c:\temp\Report\H$MathRegression_English.xml";
Report.Setup(ReportLevel.Debug, LogFile, true);
What I do eventually when I am done developing the test case is change the directory to a shared network directory then send out an email when the test is completed with a link to the directory so others in QA and DEV can read the results of the tests.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

devenb
Posts: 24
Joined: Mon Mar 09, 2009 11:50 am
Location: Bracknell, UK

Re: whitch way is the best to integrate many reports?

Post by devenb » Fri Jun 19, 2009 5:01 pm

Hey Ciege,

Thanks for your help. I did tried couple of things like mailing the rxlog alongwith the stylesheet which didnt helped.
This one has done the trick :). I am feeling like a n00b now.
Thanks a lot again.

Regards,
Deven Bairat.

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: whitch way is the best to integrate many reports?

Post by Ciege » Fri Jun 19, 2009 5:11 pm

No need to feel like a nOOb... we are all in a learning process and helping each other out (even with the easy tricks) is a way to make the community stronger.

That being said you are already one step closer to having your own test index html file like I mentioned in my first post in this thread. That way you can keep a history of all your automation tests and anyone can go in an look at them at any time. Thats the part that I would like to see you share when you get that done (just so I don't have to do it!!!!).

Have fun!
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...