Execute multiple testrunners from single testsuite

Best practices, code snippets for common functionality, examples, and guidelines.
houseofcutler
Posts: 52
Joined: Fri Mar 21, 2014 4:22 pm

Execute multiple testrunners from single testsuite

Post by houseofcutler » Sun Feb 12, 2017 6:31 pm

Hi all,

I have created a Test suite (for use with a CI system) that will interrogate multiple log files and pass this nicely out the other side of the Ranorex reporting engine. (There are about 20 log files)

In order to get 1 Ranorex report per log file I have edited the Program.cs file of the project as follows:
public static int Main(string[] args)
        {
            Keyboard.AbortKey = System.Windows.Forms.Keys.Pause;
            int error = 0;

           string current = System.IO.Directory.GetCurrentDirectory();
            string[] files = System.IO.Directory.GetFiles(current,"*.log");
	    string filename;
	    string testname;
			
        	foreach( string file in files)
        	{
        		filename = System.IO.Path.GetFileName(file);
        		testname = System.IO.Path.GetFileNameWithoutExtension(file);
        		 try
	            {
	            	            	
	            	error = TestSuiteRunner.Run(typeof(Program), "/pa:varFileName=" + filename + " /rul:" + testname + " /rf:" + testname + ".html /rl:Info");
	            }
	            catch (Exception e)
	            {
	                Report.Error("Unexpected exception occurred: " + e.ToString());
	                error = -1;
	            }
	       }
            return error; 
      }
Example output report - https://www.screencast.com/t/OTceg91IJkq

Report 1 is as expected, Report 2 includes the Rulelabel of report 1, shows 2 lots of 'TestData' and now includes double the test cases. Report 3 everything is tripled etc.

Is there anyway to make this work from within my project or do I just need to call the testsuiterunner externally from some kind of wrapper instead?

I anyone can offer any suggestions or advice I would be very grateful.

Many thanks

Ben

qwertzu
Posts: 178
Joined: Wed Jan 25, 2017 11:08 am

Re: Execute multiple testrunners from single testsuite

Post by qwertzu » Mon Feb 13, 2017 4:11 pm

Hi houseofcutler,

Do I understand you correctly?
You have about 20 log files, you want to take data out of those and write it into Ranorex reports. For each log file one report.

Why not just run each test case independently? You can setup test run configurations for each one and run them via command-line.

Afterwards, you should have a report for each of your interrogated log files.

houseofcutler
Posts: 52
Joined: Fri Mar 21, 2014 4:22 pm

Re: Execute multiple testrunners from single testsuite

Post by houseofcutler » Mon Feb 13, 2017 5:23 pm

Hi qwertzu, thank you for your response. I had considered going down that road anbd accept that would work to a degree.

I have used multiple test cases for each report in order to make the report output easier to work through. e.g.

TC1 - Results group 1
TC2 - Results group 3
TC3 - Results group 3
etc.

There are hundreds of lines of results in each log file and once filtered down, by group and errors/warning, there can still be anything up to 30 results in each Test case.

If I do one module per log file the resulting output will be a single test case with a hundred lines of results in it. which while providing the information is not as user friendly as when it is broken down into nice collapsible groups which have their own status.

qwertzu
Posts: 178
Joined: Wed Jan 25, 2017 11:08 am

Re: Execute multiple testrunners from single testsuite

Post by qwertzu » Tue Feb 14, 2017 1:57 pm

Hi,
If I do one module per log file the resulting output will be a single test case with a hundred lines of results in it. which while providing the information is not as user friendly as when it is broken down into nice collapsible groups which have their own status.
In each test case you can have multiple test modules to create log messages, which can be used to write data from your external log files into the Ranorex Report. This is a way to handle, how much information will be in the Ranorex Report and so, the reports won´t get too long.

So what you could do:
1. Create test cases for each of your external log files.
2. use Log Message actions within your test cases in order to define, which lines of your external log files should be shown in the Ranorex Report
3. Run the test cases separately in order to get a report for each case. (To be faster, you can do that via command line)

houseofcutler
Posts: 52
Joined: Fri Mar 21, 2014 4:22 pm

Re: Execute multiple testrunners from single testsuite

Post by houseofcutler » Wed Feb 15, 2017 1:38 pm

Thanks again for your suggestion qwertzu,

Your solution would work and I was going to try to implement it but I want the solution to be dynamic and cope with any log file not just a hard coded list. Apologies for not providing this as context originally .

This being the case I decided to create a wrapper to fire off the test suite form command line based on the log files it finds. This allowed me to set the program.cs file for the actual Ranorex project back to normal (no for each) and now I get the expected report content without duplication.

I didn't find this easy as outside of normal ranorex coding for logic etc I don't really do any c#. Anyway in case anyone may find this useful here is what I did...

Created an additional Project within my Ranorex solution (RanorexWrapper)
- Now both EXEs will be created when the solution is built by my CI system (teamcity)

Ranorexwrapper.exe can be executed by the CI
- Log files will be added to the directory at this point (seperate build step)

Ranorex wrapper actions:
- sets working directory and relative directory for main ranorex exe
- Runs main ranorex exe for each log file found and passes required parameters
class Program
    {
        public static int Main(string[] args)
        {
            int error = 0;

            //set working directory to wrapper exe location
            string current = AppDomain.CurrentDomain.BaseDirectory;
            Directory.SetCurrentDirectory(current);
            
            //create array of log files
            string[] files = Directory.GetFiles(current, "*.log");
            
            //set relative path to ranorex project exe
            string checkerpath = Path.Combine(current, "..\\..\\..\\XOD-Checker\\bin\\Debug\\XOD-Checker.exe");
            checkerpath = Path.GetFullPath(checkerpath);
             
			//debug info		
			Console.WriteLine("Working Dir - " + Directory.GetCurrentDirectory().ToString());
            Console.WriteLine("CheckerPath - " + checkerpath);
           
			string filename;
            string testname;
            
            foreach (string file in files)
            {
                filename = Path.GetFileName(file);
                testname = Path.GetFileNameWithoutExtension(file);
                try
                {
                    var process = System.Diagnostics.Process.Start(checkerpath,"/pa:varFileName=" + filename + " /rul:" + testname + " /rf:" + testname + ".html /rl:Info");
                    process.WaitForExit();
                 }
                catch (Exception e)
                {
                    Ranorex.Report.Error("Unexpected exception occurred: " + e.ToString());
                    error = -1;
                }

            }

            return error;
		}
    }