Anyone Want an Automation Results Dashboard Utility? : Automation Discussions

[Ciege]

I have been creating a dashboard utility for displaying automation test results. It is an ongoing process but pretty useful for us here.
Was wondering if anyone would want a copy of something like this for their use at their place of work?
If there is any interest I could probably get something out to the group by late this week or early next week after I make a couple of changes and upgrades.

Please let me know!

Functionality:
*) Will display all test results found in a convenient treeview.
*) Will allow opening the log file directly in IE.
*) Will highlight tests that have been executed within the last X number of days.
*) Optional - Will display Red/Yellow/Green for tests that have Failed/Warning/Passed.
*) Optional - Will bubble up the test results to the parent node(s).

Requirements:
*) All test results must be stored in a central location accessible by the local user (in my case all my test results are stored on a network share accessible to everyone).
*) Currently only logs converted to XML are supported.


See below for a screenshot of the current rev of the utility:

Resgression Results Dashboard

v13.jpg (94.83 KiB) Viewed 894 times

[Ciege]

OK, here it is. If interested, please give it a shot and let me know what you think.
Just extract the file and run.

v0.6 - Adds ability to find RXLOG files as well as XML files.
v0.7 - Better information displayed for access denied folders.
v0.8 - Fix if no subfolders in root folder.
v0.9 - Added config for Version Tested string in log files.
v0.10 - Added basic Report generation.
v0.11 - Added ability to export .TXT or .XML reports.
v0.12 - Added ignore OLD and RETIRED folder option to allow utility to skip old or retired folders
v0.13 - Fix for ordering test result files. Now in last write time order.
v0.14 - Added option for how to get test results when requested.
v0.15 - Added support for case insensitive log level results.

[atom]

Hiya

Hmm.... Nice idea, but its been done before

If you use visual studio say, develop your test cases using MSTest
You then already get a results dashboard

If you develop your tests using NUnit, the NUnit GUI Runner will display all results to you, and produce an xml file of all your results

If you code direct .exe's then perhaps do what we do:
- create a TestResult class
- each test case populates a TestResult class, and at the end calls a Save method
- Save, method writes the result to oracle/sql server/access, closes the ranorex log file, and copies it to a shared directory
- Develop a small web application to load results from Oracle and display then in a web page
with a link to the ranorex log file



Cheers

[Ciege]

Certainly good ideas. However, what my task is here (as the only automation person) is to check the results of a test myself to verify the results are accurate and not a bug in my automation code that caused an issue or a timeout issue, etc... Only then, after I have verified the results (as a true pass or fail) do I move the results XML file(s) to the shared directory of results for the rest of the QA team and development team to see.

[atom]

Not one for blowing my own trumpet but I did have an idea to sell such a product:

http://www.testresultx.com/

[Ciege]

Uh-Oh, did I step on your toes by giving mine away for free?

[atom]

Ahh no didnt actually start writing that anyway!

Was just an idea... as it seems most major automation players (except microsoft), forget that the most important thing about automation is the results, this is where the value of automation can be seen

If you have to scan 100's/1000's of log file to know if all tests pass its a pain
To compare current run of of tests, with previous run to see improvement is again quite painful when all you have is log files

Test managers dont care about ranorex log files to be honest
What they want to see is a gobal picture:
- how many test ran
- how many passed / failed
- is it better / worse than previous run
- what is the over-all coverage
- etc.

So to create a global picture, having all results, of all runs stored in one place, in a convienient format (Oracle, SQL Server, Access say), makes this very easy!

Anyway that was my aim with that, never actually got time to do it

[atom]

Sorry im not being critical of your tool, i think its good to share such stuff!

I just think Ranorex should do something to define what is a result, and have a way to store it, rather than us all developing our own ways

[Ciege]

No, I didn't take it that way at all... So no worries there...

However, I do agree about Ranorex providing a results dashboard thing as well. But I'm more than happy to have them deliver the strongest test solution they can at an affordable and competitive price before they delve into the results half of it. I know they are a pretty small team compares to the "big guys" and certain things need to take priority. So if I (or anyone) can provide a temporary, albeit un-polished, solution to the reporting aspect before Ranorex has time and manpower to do it I'm fine with that.

I see that my utility has been downloaded 5 times thus far but no comments on it. I just want people to give it a shot and let me know how it works out for them and if there are any additional features they would like added to it until an "official" utility is released by Ranorex. I'm more than willing to spend my copious amounts of extra time on it to make it better.

[JayDee]

Hey Ciege,

i want to try your tool but it don't detect any of my logfiles. I converted them into XML and set the paths... any idea what i did wrong?

Greetings, jd

[Ciege]

After you set the paths in Config did you click the refresh button?
Do you get a message in the app that it cannot find the directory and to set it in Tools -> Config?
Does your machine have access to the path you set?
Is the path correct in the Config.XML file that gets created?

[Ciege]

Latest version 0.5 has been added to the second post.

[Ciege]

Updated to 0.7.

[Ciege]

Updated to v0.10.
Added some fixes and a basic report generation ability.

[Ciege]

Updated to v0.11.
Added ability to export .TXT or .XML reports.