Validation message only on failure

[tmcpherson]

Is it possible to make a validation message only show up if a test fails? I'm used to more traditional assert libraries, where if the assert passes, the test runner/report is unaware of it entirely. A detailed error message doesn't even appear. With Validate methods, however, the message always appears.

Is there a Validate function I don't see that acts more like what I'd like? I can write my own otherwise, but I'd prefer to keep my custom code to a minimum. Alternatively, what sorts of customization options are there for Validate methods?

[krstcs]

You should really keep the report level at the default ("Info").


However, you can set the report level in the test suite's properties to "Failure" and it will only show Failure report log items in the report. Right-click the suite's name in the test case view and select "Properties". Then, look for the combo-box at the bottom called "Report Level:" and change it from "Info" (the default) to "Failure".


NOTE: You will lose a LOT of data about the conditions of the system before the failure, so you won't have ANY WAY to analyze what actually happened, because Ranorex won't even save those report items that are less than the level you select here.

Again, you really should keep it at "Info".

[tmcpherson]

That's a good point, and it would accomplish what I described in my post, but I agree that the report should pretty much always show most messages so that I also see the other logs I actually want to see.

In the particular case that is irritating me, I'm validating an XML response from interacting with our product's API (so this is all code, not a UI recording, in this case). In this instance, I don't really mind if there's a message, but I'd really like to customize or limit the message depending on the result. For example, if I were using MS Test's assert library (with which I'm already familiar), the following code:

Code: Select all

Assert.IsTrue(xml.Contains("someElement"), "Failed to find element 'someElement' in XML.  XML response:  " + xml);

would only print anything if the assert failed. If the assert passed (so the XML response did contain the desired element) it wouldn't even show in the output. This is desirable because the XML can be quite large, and I check it often.

Edit to add this:

I mentioned in the OP that I could write custom code for this. The code would look something like

Code: Select all

public static void ValidateExists(RepoItemInfo item, string message=null)
{
    if (!item.Exists())
    {
        var msg = "Item " + item + " exists when it should not.  " + 
            (message == null : "" ? "Message: " + message);
        Report.Error(msg);
    }
}

[krstcs]

I think that the if(exists) logic is the proper way to handle the situation you want. You can make the repo item's path be exactly what you want and then create the logic so that if it does exist (no !) or doesn't exist (!) then print the report error message.

However, your logic appears to be different than your message text in your example. The logic checks for NOT (!) exists, but the message text says it does exist when it shouldn't. But this could just be an example, so...

Maybe it should be:

Code: Select all

if (item.Exists()) { //notice, no "!"
  ...
}

[tmcpherson]

Yeah, it's kind of unfortunate. I was hoping for an action I could use like the default Validate stuff, but it doesn't seem like that's possible. Custom code it is.

Thanks for the help.

[Stub]

I too was anticipating Validate to operate in the same manner as an Assert. So I originally coded the message to be written in terms of failure only. I got a shock when I saw the failure message being passed, until I realised it was a "I'm validating" message to indicate what happened, success or failure.

[tmcpherson]

Haha, yeah. My log said things like "SUCCESS: X item was Y when it was supposed to be Z."

That said, the custom code is working as I want it to.