Need help ! Ranorex multiple test suite execution in jenkins

[Noah]

Hello,

my project has got 2 test suite files with name 1) It.rxtst 2)Itte_Environment.rxtst and in Jenkins I scheduled a job to run with
the test suite file name as"Itte_Environment.rxtst" please take a look at the image attached.

when I run a new build I get this below error.

c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns>".\trunk\Itte\bin\Debug\Itte.exe" /reportfile:"c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns\.\Reports\378\ITTE.html" /junit /zipreport /zipreportfile:"c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns\.\Reports\378\ITTE-378.rxzlog" /pa:Domain=https://10.158.118.16:28443 /tc=CancelSSN
[2018/08/21 14:03:26.461][Debug ][Logger]: Console logger starting.
[2018/08/21 14:03:26.483][Error ][User]: Unexpected exception occurred: System.InvalidOperationException: There is more than one test suite and some arguments in '".\trunk\Itte\bin\Debug\Itte_Environment.rxtst .exe" [/u]/reportfile:"c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns\.\Reports\378\ITTE.html" /junit /zipreport /zipreportfile:"c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns\.\Reports\378\ITTE-378.rxzlog" /pa:Domain=https://10.158.118.16:28443 /tc=CancelSSN' require it to specify which test suite to use by using the 'testsuite' command line parameter.
at Ranorex.Core.Testing.TestSuiteRunner.Run(Type containerType, String cmdLineArgs, String testSuiteXml)
at Ranorex.Core.Testing.TestSuiteRunner.Run(Type containerType, String cmdLineArgs)
at Itte.Program.Main(String[] args) in c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns\trunk\Itte\Program.cs:line 39
[2018/08/21 14:03:26.621][Debug ][Logger]: Console logger stopping.
Build step 'Run a Ranorex test suite' changed build result to FAILURE
[htmlpublisher] Archiving HTML reports...
[htmlpublisher] Archiving at BUILD level c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns\Reports\378 to C:\ohj\Jenkins\jobs\ITTE_Automation\jobs\ITTE_Single_Testruns\builds\378\htmlreports\ITTE
ERROR: Specified HTML directory 'c:\ohj\jenkins.start\workspace\ITTE_Automation\ITTE_Single_Testruns\Reports\378' does not exist.
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Started calculate disk usage of workspace
Finished Calculation of disk usage of workspace in 0 seconds
Finished: FAILURE

[odklizec]

Hi,

I believe you need to add a testsuite parameter to the Command line arguments section of Ranorex plugin. Please see the screenshot from my Jenkins setup. Similarly, you need to define the testsuite parameter, pointing to your testsuite. Hope this helps?

TestSuite_param.png

[Support Team]

Hi all,

@Pavel: Thank you for jumping in

Pavel's suggestion is correct, due to some changes in the latest Ranorex versions (Feature: Multiple Test Suites) it is currently necessary to add the '/ts' parameter.
I'm currently working on an update of the plugin to match the latest Ranorex version. If you are having any feedback or requests, please feel free to share them on the corresponding GitHub repo

Thanks,
Markus (S)

[odklizec]

Hi Markus, you are welcome. I'm always glad to help

[Erik Skrabak]

Hello,

I also have multiple Testsuite files in my project - let's say TS_A.rxtst and TS_B.rxtst.
In Jenkins i would like to start a job with one specific tescase (TC_X) within TS_A.rxtst only.

My Jenkins-Job configuration looks like this (see attachment).

The test fails with following exception:

"Unexpected exception occurred: Ranorex.RanorexException: There are multiple test suite files in the assembly directory. Please specify the test suite by using command line parameters or rename the test suite file to the current assembly name.
bei Ranorex.Core.Testing.TestSuiteRunner.GetTestSuiteXml(Type containerType, String fileName)
bei Ranorex.Core.Testing.TestSuiteRunner.Run(Type containerType, String cmdLineArgs, String testSuiteXml)
bei Ranorex.Core.Testing.TestSuiteRunner.Run(Type containerType, String cmdLineArgs)
bei owi21V4.Program.Main(String[] args) in C: ..."

Does anyone please have an idee how to configure this job so that only testcase TC_X from TS_A.rxtst will be startet?

Thank You!

[Support Team]

Hi Erik,

Markus (the developer of the Jenkins Ranorex Plugin) is currently working on updating this plugin to be compatible with multiple test suites.

I'm currently working on an update of the plugin to match the latest Ranorex version. If you are having any feedback or requests, please feel free to share them on the corresponding GitHub repo

In the meantime, you can use the standard "Execute Windows batch command" in Jenkins with the desired arguments. For a full list of available arguments, check out this page in our user guide ("Command line arguments" section).

Example:

Code: Select all

ProjectA.exe /ts:TestSuite2.rxtst /tc:TestCase2

1.png

I hope this helps!

Cheers,
Ned

[ggalpris]

I'm currently working on an update of the plugin to match the latest Ranorex version. If you are having any feedback or requests, please feel free to share them on the corresponding GitHub repo

Markus (S)

Hello Markus

are there any news regarding the plugin update?

Best regards Georgina

[TimoL]

I have run test cases of one test suite (let's call it as 'A') successfully in Jenkins. No I added a job from another test suite ('B' in different folder), but Jenkins says "Could not find matching test suite file for 'A'". There's no reference to test suite A in test suite B's job configuration.

Running the test case from command line succeeds (after deleting other rxtst's in the same folder): "C:\Users\xxx\Documents\Ranorex\RanorexStudio Projects\yyy\zzz\bin\Debug\B.exe". I tried also the same with Execute Windows batch command in Jenkins, but it don't find the file.

What could help? Ranorex plugin is the latest 1.0.2 version.

[N612]

Are you able to provide the Jenkins console output for this build that failed? This will hopefully contain more information on why it is failing.

[TimoL]

Finally I had time to examine this more. The problem is not in Jenkins, but in the exe file. I have under the same solution test suite A and test suite B. Test suite A has been run in Jenkins successfully for a long time.

When I try to run a test case of test suite B from command line: B.exe /tc:TC1
It leads to a failure: [Failure][TestSuite]: Could not find a 'test case' or 'smart folder' with name or ID 'TC1'

When I try to run the whole test suite from command line: B.exe
It starts to execute test suite A: [Info ][Test]: Test Suite 'A' started.

So why on earth it starts to run a wrong test suite?

[TimoL]

Problem solved by deleting the testsequence file.