I have a question about reporting to TestRail the results of a test run that iterates over entries of a data source.
This is the structure of my test suite:
(I created this structure because I am going to add an "Import" folder to the test suite, and more test cases to smart folders A and B.)
Code: Select all
+ test suite "ImportExport" +- smart folder "Export" +-- smart folder "Export A" | +-- test case 1 +-- smart folder "Export B" +-- test case 2
The smart folder "Export" contains a simple data source with two rows. Test case 1 contains a condition so that is executed for data row 1 only. When I run the suite manually, I am reported "3x Success, 1x Blocked" which comes close to what I would expect ("1x Ignored" would be perfect).
In TestRail however, I see "2 Passed"; the results of both runs of test case 2 are merged into one report, and there is no indication that one run of test case 1 was skipped. The latter information is important because it reminds me to fix a bug in the AUT which destroys the test data (and that is why I skip the run, rather than letting it fail).
Am I doing something wrong, or is that normal behaviour? Any ideas how to solve the issue?