Continuation of the topic about reports

Ask general questions here.
taralex
Posts: 57
Joined: Tue Nov 04, 2008 3:50 pm
Location: Michigan
Contact:

Continuation of the topic about reports

Post by taralex » Tue Jan 31, 2012 6:22 pm

I'm going to start a new thread since I'm not really sure if the way I described the problem initially is correct.
So I'll start from the beginning and try to give all the code and screenshots. So the post is gonna be long, I'm sorry.
Here's the function I'm calling:
public void RunTests(string[] tsts)
        {
            
            //report folder will be prepared regardless of if it's UI mode or not. 
            //the report will be shown at the end of a run of any set of tests.
            string reportPath = Path.Combine(Application.StartupPath, "Reports");
            string date = System.DateTime.Now.Month + "-" + System.DateTime.Now.Day + "-" + System.DateTime.Now.Year +"-" + System.DateTime.Now.Hour + "-" + System.DateTime.Now.Minute + "-" + System.DateTime.Now.Second;
            CurrentReportFolder = Directory.CreateDirectory(Path.Combine(reportPath, date));
            File.Copy(Path.Combine(Application.StartupPath, "RanorexReport3.png"), Path.Combine(CurrentReportFolder.FullName, "RanorexReport3.png"), true);
            File.Copy(Path.Combine(Application.StartupPath, "AVLLogo.png"), Path.Combine(CurrentReportFolder.FullName, "AVLLogo.png"), true);
            File.Copy(Path.Combine(Application.StartupPath, "AVLReportTemplate.xsl"), Path.Combine(CurrentReportFolder.FullName, "AVLReportTemplate.xsl"), true);
            string reportFilePath = Path.Combine(CurrentReportFolder.FullName, "AutomatedTestsReport.xml");

            
            
           // Report.Setup(ReportLevel.Info, Path.Combine(CurrentReportFolder.FullName, "AutomatedTestsReport.xml"), true);
            TestReport.Setup(ReportLevel.Info, Path.Combine(CurrentReportFolder.FullName, "AutomatedTestsReport.xml"), true);
            TestReport.StyleSheetFilename = Path.Combine(CurrentReportFolder.FullName, "AVLReportTemplate.xsl");
            TestReport.ReportWriteInterval = 0;                                    
            TestReport.BeginTestSuite("Automated Tests");
            foreach (string item in tsts)
            {
                foreach (AutomatedTest tst in LoadedTests)
                {
                    string itemWithoutTestcases = item.Remove(item.IndexOf(" ("));
                                      
                    if (itemWithoutTestcases == tst.Name)
                    {
                        // Assembly assemblyToRun;
                        //assemblyToRun = GetAssemblyByID(tst[0].ToString());
                          
                        if (tst.CompiledAssembly == null)
                            return;
                        Type[] types = tst.CompiledAssembly.GetTypes();
                        foreach (Type t in types)
                        {

                            foreach (object attribute in t.GetCustomAttributes(true))
                            {
                                Type a = attribute.GetType();
                                if (a.Name == "ClassInfoAttribute")
                                {
                                    object instance = Activator.CreateInstance(t);
                                    MethodInfo meth = t.GetMethod("StartTest");
                                    if (meth != null)
                                    {                                        
                                        string codeFileInRuntime = Path.Combine(Application.StartupPath, Path.GetFileName(tst.Path));
                                        if (File.Exists(codeFileInRuntime))
                                            File.SetAttributes(Path.Combine(Application.StartupPath, Path.GetFileName(tst.Path)), FileAttributes.Normal);
                                        if (tst.Path != Path.Combine(Application.StartupPath, Path.GetFileName(tst.Path))) //will crash if it's the same file
                                            File.Copy(tst.Path, Path.Combine(Application.StartupPath, Path.GetFileName(tst.Path)), true);
                                        TestReport.BeginTestCase(item);
                                        try
                                        {
                                            
                                            object retVal = meth.Invoke(instance, null);                                           
                                            if ((int)retVal == 1)                                            
                                                TestPassedActions(tst);                                            
                                            else                                            
                                                TestFailedActions(tst);
                                            
                                        }
                                        catch (Exception e)
                                        {
                                            if (e.InnerException != null)
                                            {
                                                WriteLog(e.InnerException.Message);                                                
                                                
                                                //if in the test case file, the user started a test module, but it didn't finish due to an exception within the test
                                                //we'll finish it here to preserve the report integrity
                                                if (TestReport.CurrentTestModuleActivity != null)
                                                {
                                                    Report.Failure(e.InnerException.Message);
                                                    TestReport.EndTestModule();
                                                }
                                                TestFailedActions(tst);
                                            }
                                        }
                                        TestReport.EndTestCase();

                                    }
                                    else
                                    {
                                        WriteLog("Couldn't locate entry point in the loaded test.");
                                        if (this.UImode)
                                            MessageBox.Show("Couldn't locate entry point in the loaded test.\r\n Make sure your test contains a StartTest method returning 1 for success and 0 for failure.");
                                    }
                                }
                            }
                        }

                    }                    
                    
                    
                }
            }            

            
            TestReport.SaveReport();            
            Process htmlInDefaultBrowser = new Process();
            htmlInDefaultBrowser.StartInfo.UseShellExecute = true;
            htmlInDefaultBrowser.StartInfo.FileName = Path.Combine(CurrentReportFolder.FullName, "AutomatedTestsReport.xml");
            htmlInDefaultBrowser.Start();
            SetTestsRunning(false);
            //ActivityStack.Clear(); 
        }
I edit the test code that I load to succeed (return 1) and call this function. At this point I'm seeing the correct report at the end.
Then I edit my test code to fail (return 0) and call this function again (without unloading my test script exe). Here's when things become interesting, the ActivityStack looks correct:
activityStack.PNG
activityStack.PNG (127.28 KiB) Viewed 1530 times
However, when I press F5 to continue, here's what is shown in the report:
report.PNG
report.PNG (37.67 KiB) Viewed 1530 times
Now if I uncomment the ActivityStack.Clear(); line at the end of the function, call it once (see the correct report), then call it again and see the following:
report2.PNG
report2.PNG (17.71 KiB) Viewed 1530 times

I was fighting with this thing for several days now, and it's probably something insanely simple. Can anyone stick my face into what I'm doing wrong?

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Continuation of the topic about reports

Post by Support Team » Fri Feb 03, 2012 1:16 pm

Hi,

Thank you for posting us this information, but in your case it would be useful to get the entire project/solution. Would this be possible?

Did you also check the Report File (XML View) if the test case status is saved correctly?

Regards,
Peter
Ranorex Team
.
Image

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Continuation of the topic about reports

Post by Support Team » Tue Feb 07, 2012 4:27 pm

Hi,

we have had a look at your code.
It looks like there where some threading issues with using the activitiy stack.
As workaround you can use following code snippet before setting up your report which make sure that the right Activity Stack will be taken for your reporting:
typeof(TestReport).GetField("targetThread", BindingFlags.Static | BindingFlags.NonPublic).SetValue(null, Thread.CurrentThread);
ActivityStack.Clear();
We will discuss this behavior internally and will take account of it in our road map.

Regards,
Tobias
Ranorex Support Team
.
Image

taralex
Posts: 57
Joined: Tue Nov 04, 2008 3:50 pm
Location: Michigan
Contact:

Re: Continuation of the topic about reports

Post by taralex » Tue Feb 07, 2012 7:11 pm

It worked, thank you.

This is probably another topic, but I noticed a strange behavior on a Windows7 machine. My application exits on calling TestReport.SaveReport(); there is no exception, it looks like Environment.Exit(); is called somewhere in the Ranorex code. I'm running it as Administrator and it doesn't happen on XP computers...

Do you have an idea why this may happen?

User avatar
Support Team
Site Admin
Site Admin
Posts: 11709
Joined: Fri Jul 07, 2006 4:30 pm
Location: Graz, Austria

Re: Continuation of the topic about reports

Post by Support Team » Wed Feb 08, 2012 12:37 pm

Hi,

I'm sorry but we were not able to reproduce this behavior.
Basically Environment.Exit() will only be triggered if there is no valid license, which is not the case in your scenario.

The only thing we can imagine is that you have some unhandled exceptions within your code.

Regards,
Tobias
Ranorex Support Team
.
Image

taralex
Posts: 57
Joined: Tue Nov 04, 2008 3:50 pm
Location: Michigan
Contact:

Re: Continuation of the topic about reports

Post by taralex » Wed Feb 08, 2012 8:33 pm

Oh, that what may be the problem! I've got a floating license and it may have happened that I tried using Ranorex on one machine while it was in use on another. that is what happened I guess, thanks, I almost scratched a hole in my head figuring out what's happening.
I guess you should show some message box if there is license problem, it's kinda frightening when your application just quits.