Anyone Want an Automation Results Dashboard Utility?

Experiences, small talk, and other automation gossip.
User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Tue Feb 01, 2011 7:25 pm

Updated to v0.13.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Fri Mar 25, 2011 4:06 pm

I think Ranorex has done a fine job so far providing the reports that they provide. The beauty of the complete Ranorex toolset is the openness and the ability for the end user base to extend it in any way that we see fit.
The way I present my automated test results to my team and management is different than the result reports that Ranorex provides, thus the dashboard utility that I wrote. Because they provide their reports in XML we are not tied into using only the reporting mechanism that they have given us, which in my mind is a better solution than most of the "big name" automation tool providers that give you results in a closed format that requires their own specific (and usually licensed at a extra cost for each user) report viewer and dashboard.
Just my 2 cents...
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

KAB
Posts: 10
Joined: Mon Jan 09, 2012 1:30 pm

Re: Anyone Want an Automation Results Dashboard Utility?

Post by KAB » Thu Jan 19, 2012 10:11 am

Hi

The tool looks really usefull, but I have a few questions (after having tried it)

- For all my .rxlog-files, i get "RESULT: UNKNOWN", so what am I missing? Do I need to write some kind of "test result verdict" into my log files for the tool to be able to give the overview?

- In the Tool there is reference to a "Test Plan", what is that? (I'm new to automated testing, so bear with me if it's something I should have been able to figure out myself)

Regards
/KAB

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Thu Jan 19, 2012 3:39 pm

KAB wrote:The tool looks really usefull, but I have a few questions (after having tried it)
Thanks.
KAB wrote: - For all my .rxlog-files, i get "RESULT: UNKNOWN", so what am I missing? Do I need to write some kind of "test result verdict" into my log files for the tool to be able to give the overview?
The code looks at the markup of your results files specifically for the following entries to determine the overall result of the test. If it cannot find any of these it will default to a Result of Unknown. These are all statuses that are set by Ranorex when you issue an Error, Failure, Warning or Success message in the log.

Code: Select all

level=\"ERROR\"
level=\"FAILURE\"
level=\"WARN\"
level=\"SUCCESS\"
KAB wrote: - In the Tool there is reference to a "Test Plan", what is that? (I'm new to automated testing, so bear with me if it's something I should have been able to figure out myself)
Since this is a tool we use everyday here at my work, I was asked to put in a button that opens a Windows Explorer to our network location that holds our test plans. You can assign that location in the Tools -> Config dialog under the "Path to test plan folder" text box.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

KAB
Posts: 10
Joined: Mon Jan 09, 2012 1:30 pm

Re: Anyone Want an Automation Results Dashboard Utility?

Post by KAB » Tue Jan 31, 2012 1:02 pm

Thanks

Having finally gotten around to do a bit more testing with this, it seems that my problem/issue was that the dashboard is case sensitive when it looks for the levels so all my

Code: Select all

level="Error"
level="Failure"
level="Warn"
level="Success"
were not detected, as they for some reasons were not in all capital letters, but a quick find'n'replace on a few files made them appear with a result in the tool.

Now I just need to find the settings in Ranorex to make them all capital letters, and then see if this Dashboard is what the managers had in mind :mrgreen:

but thx again.

/KAB

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Tue Jan 31, 2012 8:10 pm

Thanks for that information... I've made a change in the utility to be case insensitive when looking for report log levels.
You can download v0.15 from the second post.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

omayer
Posts: 458
Joined: Thu Oct 28, 2010 6:14 pm

Re: Anyone Want an Automation Results Dashboard Utility?

Post by omayer » Wed Feb 01, 2012 9:00 pm

Hi Ciege,
Would you please provide steps how to use your dashboard utility
Thank you
Tipu

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Wed Feb 01, 2012 10:59 pm

Just extract the zip and run the EXE.

Under Help there are instructions. But basically go to tools -> config and set the path for the root of your test results folder then click the refresh button in the main dialog. There are other options that are (mostly) explained in the help.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

omayer
Posts: 458
Joined: Thu Oct 28, 2010 6:14 pm

Re: Anyone Want an Automation Results Dashboard Utility?

Post by omayer » Thu Feb 02, 2012 4:25 pm

Thank you Ciege, will give a try and keep you posted
Tipu

omayer
Posts: 458
Joined: Thu Oct 28, 2010 6:14 pm

Re: Anyone Want an Automation Results Dashboard Utility?

Post by omayer » Thu Feb 02, 2012 5:24 pm

Hi Ciege,
Thank you for sharing the dashboard utility, I see the result open w/.rxlog but how do opne with IE or view in IE.
Thanks,
beginner
Tipu

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Thu Feb 02, 2012 5:27 pm

As written on the help screen:
You can drill down through the tree view to the XML results log and open the log in IE by double clicking it or selecting it and clicking the View Log button.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

omayer
Posts: 458
Joined: Thu Oct 28, 2010 6:14 pm

Re: Anyone Want an Automation Results Dashboard Utility?

Post by omayer » Thu Feb 02, 2012 5:43 pm

i don't see the result in a tree view, i have to manually open each report, am i missing anything, here is the config setup
Report.jpg
Report.jpg (67.07 KiB) Viewed 1630 times
Tipu

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Thu Feb 02, 2012 5:48 pm

On the main dialog click the checkbox that says "Display test results and Version tested" then click the refresh button.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...

omayer
Posts: 458
Joined: Thu Oct 28, 2010 6:14 pm

Re: Anyone Want an Automation Results Dashboard Utility?

Post by omayer » Thu Feb 02, 2012 6:10 pm

Thank you Ciege for quick reply, its worked except one item i am missing
Report.jpg
Report.jpg (52.54 KiB) Viewed 1625 times
Tipu

User avatar
Ciege
Ranorex Guru
Posts: 1335
Joined: Thu Oct 16, 2008 6:46 pm
Location: Arizona, USA

Re: Anyone Want an Automation Results Dashboard Utility?

Post by Ciege » Fri Feb 03, 2012 6:31 pm

Hmmm, strange indeed... That is supposed to be the name of the folder that holds your testcases. In your case, whatever the folder name is under c:\SVNViews\Automation\TestSuite\bin\Debug.
Is there a folder there or is that just where your test results files live?

The way I have mine setup is like this...
\\networkshare\QAAutomation\RegressionTestResults\testfolder1\results
\\networkshare\QAAutomation\RegressionTestsResults\testfolder2\results
\\networkshare\QAAutomation\RegressionTestsResults\testfolder3\results
etc...

I set the root in the config of the utility to: \\networkshare\QAAutomation\RegressionTestsResults

Therefore each testfolder1/2/3/etc... is the name that you are missing in your view.
So it doesn't know what to put there since you do not have a folder that is named to hold the results of your tests...


One more thing... try taking out the trailing "\" at the end of debug in the config line... It should at least name that folder Debug in the view.
Last edited by Ciege on Wed Feb 08, 2012 11:21 pm, edited 1 time in total.
If this or any response has helped you, please reply to the thread stating that it worked so other people with a similar issue will know how you fixed your issue!

Ciege...