How to add multiple Jenkins jobs with a Ranorex Project

Ask general questions here.
mto
Posts: 25
Joined: Tue Jan 06, 2015 10:31 am

How to add multiple Jenkins jobs with a Ranorex Project

Post by mto » Wed May 20, 2015 11:54 am

Hello all,

I would like to add multiple jobs to Jenkins to run separately my tests.

I explain:
I have a Ranorex Project with multiple Test suites and what i would like is to execute the test suites separately.

Example:
- I have a TestSuite which is testing GUI
- I have another TestSuite which is testing browsers.

Currently, i execute all the tests at the same time. i created a job in Jenkins with MSBUILD and this BUILD is responsible for executing all the tests.

So, how i can do to separate my tests suites in multiple jobs?
- A job which is running GUI TestSuite
- Another job which is running Browsers TestSuite

I hope my topic is clear

Thank you in advance for your help.

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by krstcs » Wed May 20, 2015 3:50 pm

First, you need to understand that Ranorex creates a .NET executable for EACH project/test suite in your solution. All you need to do is run that executable file. You only need MSBUILD to BUILD the solution, not to run the tests.

Second, you can make the jobs in Jenkins run the created executable and include any command-line parameters that you need.
Shortcuts usually aren't...

mto
Posts: 25
Joined: Tue Jan 06, 2015 10:31 am

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by mto » Wed May 20, 2015 4:56 pm

Thank you krstcs for you reponse and your explanations.

Sorry, i have only one TestSuite in my project so i have one executable file ".exe".
and in this Test Suite, i have several folders (i separated my tests by using folders)
One folder is for example used for GUI tests and another is used for Browsers tests.

So which type of command can i use to separate my tests in two jobs in Jenkins?

Thank you for your help,

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by krstcs » Wed May 20, 2015 6:32 pm

I'm not sure I understand what you mean by 'folders.' Do you mean Test Cases inside the Suite, or are you talking about file folders inside the project/solution?

If by 'folder' you mean Test Cases, you can setup different configurations based on which cases you check/uncheck. Please read the user guide, it will help you understand how Ranorex works and what you can do with it: http://www.ranorex.com/support/user-guide-20.html

If you are talking about file folders in the project, you should never have more than one Test Suite file (*.rxtst) inside a given project, even if you are using folders (you said you only have 1, so that's good). You should always create new projects for each test suite you want to make. I do this all the time, each test has it's own project in the solution and when you compile, Ranorex creates executables for EACH test. This makes it easier to run each test in Jenkins separately. But, I have a very complex structure for my tests that require multiple types of tests and I found that separate suites is easier to manage.
Shortcuts usually aren't...

mto
Posts: 25
Joined: Tue Jan 06, 2015 10:31 am

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by mto » Thu May 21, 2015 9:07 am

Thank you for your response.

I use folders, this means that i separated my different test cases by folders.
If you right clic on the Test Suite you can create folders. So i used that and in each folder
i created several TestCases.

If i understand your proposal, the solution is to create different Test Suites (projects)? and when i compile, this will generate different ".exe" files. So in Jenkins i can separate my tests. am i right?

Thank you

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 3930
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by odklizec » Thu May 21, 2015 9:46 am

Hi,

If you don't want to create multiple test suites, you can use the "Run Configurations" approach mentioned by krstcs...
http://www.ranorex.com/support/user-gui ... html#c3019

Simply, you have to create different run configurations for tests you want to run and check/uncheck the appropriate test cases. Then in Jenkins, you can run these configurations separately by calling exe with "cfg" command line parameter and appropriate configuration name:
http://www.ranorex.com/support/user-gui ... html#c4827
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

mto
Posts: 25
Joined: Tue Jan 06, 2015 10:31 am

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by mto » Thu May 21, 2015 1:37 pm

thank you odklizec

this is what i was looking for. thank you very much
i used run configurations and i ckecked/unchecked tests in each configuration.

But now i have another issue with Jenkins. i separated my tests using paramater "myProject.exe /cfg:<run configuration name>".

I have only one project, and in this project exists only one file named "myProject.rxtst"
Each time i launch a build, the same "myProject.rxtst" is copied to the workspaces of my 3 separated jobs.
consenquently, when i launch the 3 builds it using the same "myProject.rxtst" so it launch the same tests each time.

i hope it is clear

mto
Posts: 25
Joined: Tue Jan 06, 2015 10:31 am

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by mto » Thu May 21, 2015 2:40 pm

thank you all.

finally i find a solution. thank you all for your help.

in fact, i use 3 different "myProject.rxtst". i use copy command to copy it in "bin\debug\" each time before launching the build and now it s ok.

NewLearner
Posts: 29
Joined: Wed Jan 15, 2014 3:32 pm

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by NewLearner » Wed May 27, 2015 10:14 am

This is general question on Jenkins, I never used Jenkins, I just read it is continuous integration tool, if so what's the benefit of using Jenkins in Ranorex? can we use jenkin as version control system?, for example if more than one person using same object repository in same time in different machine, is the jenkins helps to avoid object conflict?

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by krstcs » Wed May 27, 2015 2:21 pm

1. Jenkins is not a version control system, it is a continuous integration system, which means it builds/deploys/runs you tests. It basically is a server that controls several slave systems and can then run command-line or build level jobs on the slaves (or the server, it's the master system).

2. Ranorex has no concept of remote or parallel execution, so you use a CI system like Jenkins to build your tests, copy them to the systems where they will run, and then run the tests with any number of command-line switches, as needed. This allows you to have a full environment of test systems without having to start your test manually on each one. It also allows you to have one place where you build the test executables. You could do this with PSExec and batch files, but Jenkins is much easier and presents you with the results in one screen. I also publish my Ranorex reports as HTML files to the Jenkins web server so I can go back into any build and see the Ranorex reports for that build.

3. Jenkins can look at your version control system and know when it needs to build your projects based on commits to the version control system. I use GitHub, so I have Jenkins check GitHub for changes every 2 minutes to see if there are changes. This means I never have to manually run the build jobs. And Jenkins will only build if it sees changes, it won't build every 2 minutes.

4. You can setup Jenkins jobs to run on a schedule (I run my main tests at night while no one is manipulating the SUT), or to run after a certain event. For example, if your developers are using Jenkins you can have your tests build and run after their development jobs build and deploy the SUT to your test environment.

5. Jenkins has a concept of matrix builds, where you tell Jenkins what each configuration is and it will run the build on each configuration of slave. For example, say you need to run a web test on IE 8/9/10/11, FF, and Chrome. You can setup each slave with labels that tell Jenkins what is available on that slave (like "Windows7 RanorexExec Firefox IE09" and another like "Windows7 RanorexExec Chrome IE10"). Then you setup your matrix job to run on each browser separately. You can also filter that list some by telling the job that Windows8 only has IE10 and Windows8.1 only has IE11, so it won't try to run the IE09 job on Windows8.1, for example.


There are other systems that do the same thing, such as Cruise-Control.NET (CC.NET), which is like Jenkins but is built in .NET instead of Java. There is also Bamboo, which is what we are switching too, but it isn't free, you have to pay for each place that a slave runs, but the server is free (which for testing like we do, is probably not what you want).

Anyway, hope that helps.
Shortcuts usually aren't...

NewLearner
Posts: 29
Joined: Wed Jan 15, 2014 3:32 pm

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by NewLearner » Wed May 27, 2015 3:59 pm

Thank you for much details about Jenkins, I think I don't need this, because I'm using HP ALM for batch run in remote system(night time) scheduled one, as you said parallel run in different slaves using jenkins, here slave means different system right?, each system consider as slave, which means you can't run two different test in same machine same time right?

krstcs
Ranorex Guru
Posts: 2683
Joined: Tue Feb 07, 2012 4:14 pm
Location: Austin, Texas, USA

Re: How to add multiple Jenkins jobs with a Ranorex Project

Post by krstcs » Wed May 27, 2015 4:20 pm

That is correct. A slave is ONE (1) system in Jenkins. (A Jenkins slave CAN run multiple jobs at the same time, if you set it up that way, but with Ranorex tests, you should not do that.)

If you are using HP ALM, then you don't need Jenkins. I believe HP's ALM CI system is built on HUDSON (I could be wrong though), from which Jenkins is forked.
Shortcuts usually aren't...