Anyone Want an Automation Results Dashboard Utility? : Automation Discussions - Page 2

[Ciege]

Updated to v0.13.

[Ciege]

I think Ranorex has done a fine job so far providing the reports that they provide. The beauty of the complete Ranorex toolset is the openness and the ability for the end user base to extend it in any way that we see fit.
The way I present my automated test results to my team and management is different than the result reports that Ranorex provides, thus the dashboard utility that I wrote. Because they provide their reports in XML we are not tied into using only the reporting mechanism that they have given us, which in my mind is a better solution than most of the "big name" automation tool providers that give you results in a closed format that requires their own specific (and usually licensed at a extra cost for each user) report viewer and dashboard.
Just my 2 cents...

[KAB]

Hi

The tool looks really usefull, but I have a few questions (after having tried it)

- For all my .rxlog-files, i get "RESULT: UNKNOWN", so what am I missing? Do I need to write some kind of "test result verdict" into my log files for the tool to be able to give the overview?

- In the Tool there is reference to a "Test Plan", what is that? (I'm new to automated testing, so bear with me if it's something I should have been able to figure out myself)

Regards
/KAB

[Ciege]

The tool looks really usefull, but I have a few questions (after having tried it)

Thanks.

- For all my .rxlog-files, i get "RESULT: UNKNOWN", so what am I missing? Do I need to write some kind of "test result verdict" into my log files for the tool to be able to give the overview?

The code looks at the markup of your results files specifically for the following entries to determine the overall result of the test. If it cannot find any of these it will default to a Result of Unknown. These are all statuses that are set by Ranorex when you issue an Error, Failure, Warning or Success message in the log.

Code: Select all

level=\"ERROR\"
level=\"FAILURE\"
level=\"WARN\"
level=\"SUCCESS\"

- In the Tool there is reference to a "Test Plan", what is that? (I'm new to automated testing, so bear with me if it's something I should have been able to figure out myself)

Since this is a tool we use everyday here at my work, I was asked to put in a button that opens a Windows Explorer to our network location that holds our test plans. You can assign that location in the Tools -> Config dialog under the "Path to test plan folder" text box.

[KAB]

Thanks

Having finally gotten around to do a bit more testing with this, it seems that my problem/issue was that the dashboard is case sensitive when it looks for the levels so all my

Code: Select all

level="Error"
level="Failure"
level="Warn"
level="Success"


were not detected, as they for some reasons were not in all capital letters, but a quick find'n'replace on a few files made them appear with a result in the tool.

Now I just need to find the settings in Ranorex to make them all capital letters, and then see if this Dashboard is what the managers had in mind

but thx again.

/KAB

[Ciege]

Thanks for that information... I've made a change in the utility to be case insensitive when looking for report log levels.
You can download v0.15 from the second post.

[omayer]

Hi Ciege,
Would you please provide steps how to use your dashboard utility
Thank you

[Ciege]

Just extract the zip and run the EXE.

Under Help there are instructions. But basically go to tools -> config and set the path for the root of your test results folder then click the refresh button in the main dialog. There are other options that are (mostly) explained in the help.

[omayer]

Thank you Ciege, will give a try and keep you posted

[omayer]

Hi Ciege,
Thank you for sharing the dashboard utility, I see the result open w/.rxlog but how do opne with IE or view in IE.
Thanks,
beginner

[Ciege]

As written on the help screen:

You can drill down through the tree view to the XML results log and open the log in IE by double clicking it or selecting it and clicking the View Log button.

[omayer]

i don't see the result in a tree view, i have to manually open each report, am i missing anything, here is the config setup

Report.jpg (67.07 KiB) Viewed 69 times

[Ciege]

On the main dialog click the checkbox that says "Display test results and Version tested" then click the refresh button.

[omayer]

Thank you Ciege for quick reply, its worked except one item i am missing

Report.jpg (52.54 KiB) Viewed 65 times

[Ciege]

Hmmm, strange indeed... That is supposed to be the name of the folder that holds your testcases. In your case, whatever the folder name is under c:\SVNViews\Automation\TestSuite\bin\Debug.
Is there a folder there or is that just where your test results files live?

The way I have mine setup is like this...
\\networkshare\QAAutomation\RegressionTestResults\testfolder1\results
\\networkshare\QAAutomation\RegressionTestsResults\testfolder2\results
\\networkshare\QAAutomation\RegressionTestsResults\testfolder3\results
etc...

I set the root in the config of the utility to: \\networkshare\QAAutomation\RegressionTestsResults

Therefore each testfolder1/2/3/etc... is the name that you are missing in your view.
So it doesn't know what to put there since you do not have a folder that is named to hold the results of your tests...


One more thing... try taking out the trailing "\" at the end of debug in the config line... It should at least name that folder Debug in the view.