Code examples for post analysis reporting

[bygones]

Hi,

I like to write a plugin/extension to report test execution results to external services. I struggle quite bit with the lack of documentation (or me not finding it) about how to do this.

In my case I like to retrieve information such as

- how many test cases were successful/failed in my test suite
- how long took each case or how long took the whole suite

in general information about the test execution and its result.

Can you provide me with some information how to accomplish this ?
I'm using Ranorex 5.4.3.26106

[Support Team]

Hello bygones,

I’m afraid that there is no easy way to retrieve the needed information. You would need to use internal, undocumented API, which can change within every new release of Ranorex. So this wouldn’t be a very reliable way. I would suggest implementing your own logging mechanism in order to log such information.

Thank you for your understanding.

Sincerely,
Robert

[bygones]

This would increase the value of the API enormously. I hope this is considered somewhere in the future

[odklizec]

I'm not sure what you mean? What should be considered in future? If you mean that you want to set this value from code, then I'm sure it's already possible (check the API). I just don't see what's the point of setting it from code, if you want to use recorder? Setting things from code makes sense only for coded tests (i.e. without using recorder).

[jasoncleo]

Hi Bygones,

How we addressed this ourselves was to leverage .Net to do our own processing.

We use Jenkins to run all our test in a CI environment, that ensures that all the reports (rxlogs) get poured into a single common folder.

The actual results are all in the rxlog.data and this is just an XML format file. A simple XSLT can convert this into a rough JUnit report if you want to get results into Jenkins, but that isn't the greatest as JUnit treats every step/module as a test.

You can do your own post processing of the XML just using the .Net System.Xml libraries to traverse the nodes and get the information you need to pass onto your other systems. This probably works out the better option, as that way you can refine the type and level of data as you need.

Code: Select all

using System.Xml;
.
.
.
XmlDocument xDoc = new XmlDocument();
xDoc.Load(your rxlog.data file here);

// Define our own namespace as the rxlog.data doesn't have one
XmlNamespaceManager ns = new XmlNamespaceManager(xDoc.NameTable);
ns.AddNamespace("x", xDoc.DocumentElement.NamespaceURI);

// Getting total fail/error counts. Note whatever namespace you added, you should use here.
var node = xDoc.SelectSingleNode("/x:report/x:activity", ns);

// You should test the result of the TryParse or something. I'm just hacking this together here.
int countErrors = 0;
int countFailures = 0;
int.TryParse(node.Attributes["totalfailedcount"].Value, out countFailures);
int.TryParse(node.Attributes["totalerrorcount"].Value, out countErrors);

Some other attributes you can get from that node are:

• "host" - For the host machine that ran the test if you use a distributed execution environment.
  "timestamp" - For the local start time when the test began.

Going to a different node: /report/activity/activity (don't forget to put in your namespace), you can get:

• "duration" - For the time that the test suite took to run. You may need to do some processing as this format changes based on whether it was less than a minute, and even less than a second I think.
  "testsuitename" - Gives you the name of that test suite (in the event that your reports don't contain the name of the test suite, which may happen as that can be overridden at the commandline).

All up, just play around to get the info that you need.

[bygones]

@odklizec
I don't want to set anything for a test run, the idea here is to use a finished test run and retrieve information from this run to create individual reports (sending a hipChat message or whatsoever).

@jasoncleo
thanks for the post - I will take a look at the xml

[jasoncleo]

No worries Bygones.

Speaking of HipChat, we use that too. We set up a background process that runs on the same server as our Ranorex license manager and it posts on a number of boards with license status, test results and Ranorex UI Repository lock status.

If you need a good base program, then I recommend the XmppBot off GitHub (Google for XmppBot HipChat and you'll find it).

Cheers,
Jason