Number maximum of test suites

Ask general questions here.
User avatar
Julien Diot
Posts: 59
Joined: Wed Feb 20, 2019 4:25 pm
Location: Amiens (FR)

Number maximum of test suites

Post by Julien Diot » Tue Jul 30, 2019 1:28 pm

Hello,

I would like to know if there was a recommended maximum number of test suites and a recommended number of test boxes for a test suite?

I ask myself this question because when I execute a test case by a test case, my test cases end in success while these same scenarios can end in failure if I execute it by running my complete test suite.

For information, my test suite contains 22 test cases and runs in about 4 hours.

I also have a test suite of 75 test cases that when I launch it, I end up with my RANOREX project that takes 99% of the process...

Thank you for your feedback

User avatar
Stub
Posts: 515
Joined: Fri Jul 15, 2016 1:35 pm

Re: Number maximum of test suites

Post by Stub » Tue Jul 30, 2019 2:25 pm

We have 5 or 6 Test Suites in our test solution now. Tons of smart folders, and in one case I know I have almost 300 Test Cases in one single Test Suite that takes ~2h to run. Ranorex have previously stated that their overnight tests take about 12 hours.

I don't think your small number of Test Cases is the problem, it's more likely your test structure/test flow/AUT IMO. I know in our tests that I've put in quite some effort to make them run sequentially in bulk, but also in selective blocks (organised via smart folders) and individually when I'm examining test failures.

User avatar
Julien Diot
Posts: 59
Joined: Wed Feb 20, 2019 4:25 pm
Location: Amiens (FR)

Re: Number maximum of test suites

Post by Julien Diot » Tue Jul 30, 2019 3:08 pm

Thank you for your feedback.

That's what I thought the problem was on my side.
I just wanted to get an outside opinion :D

I'm not the one who's automated all these tests, I took over the project not long ago, so I'll try to find out what's wrong with the structure :P