Cleanup after failing tests

[boegvald]

How do you cleanup automatically after failing (browser-based) tests?

This theme seems to be an endless one for me. A test can fail in so many ways, leaving browsers open with open modal dialogues, file dialogues, etc. For now, I have ended up with starting all tests with trying to close all existing browsers, then killing any browser processes left. But some times that leaves some browsers (like Chrome) to start up with warnings, which then have to be handled… It feels like a never ending challenge…

And yes, I know, tests should always leave the environment intact. But they doesn't

Do you have an effective way of cleaning up? Are you using some kind of sandboxing or virtualization to start from scratch each time?

All good ideas welcomed…

(Primarily doing browser based tests, stored In GitLab, run on a virtual server via Jenkins).

Best Regards
Bo

[RobinHood42]

Hi,

There is no sandboxing or virtualization in Ranorex. It simply tests your UI. If your test ends up with open browsers I assume the test doesn't work correctly
There seem to be some edge cases you didn't implement in your test scenario. If you know which dialogs are open, you can close them the teardown section.

Cheers,
Robin

[odklizec]

Hi,

One thing you can implement, and which I'm successfully using for one of my TA projects, is creating a VM snapshot, just before the test starts, run the test and then, regardless of test outcome (success of failure), return to the snapshot state. And because the snapshots could become quite big and clutter the VM storage, always delete the snapshot. This approach could help you with opened dialogs, improperly closed windows and changes done in system. What it will not help you with, are potential DB changes, unless the DB is stored locally. Otherwise, you must find a way to restore DB back to its previous state. Hope this helps a bit?

[boegvald]

Good to see that you have been successful at using snapshots to 'reset' the testing environment. Which technologies are you using?

I was considering a variant of your setup, creating a snapshot once, then starting each test with returning to the snapshot. But your approach seems somewhat more flexible (systems updates and other environment changes are easier to handle).

But I can find no information about using Jenkins to work with Hyper-v snapshots… Maybe there is a plug-in I just can't find? Or it's just a matter of writing a couple of power shell scripts? Maybe not?

Restoring the DB is no problem as I already have a way of doing that.

Best Regards
Bo

[odklizec]

Hi,

I'm using vSphere plugin in Jenkins, which is configured to perform all snapshot-related steps. Unfortunately, I have no experience with Hyper-v. If there is no Hyper-v plugin in Jenkins, then you may need to use command line/powershell to create/restore Hyper-v snapshots? Check for example this blog post:
https://devblogs.microsoft.com/scriptin ... owershell/
And also this one:
http://hyper-v-backup.backupchain.com/h ... snapshots/
Hope this helps a bit?

[boegvald]

Thanks much for the 'hints'. It doesn't look too complex. I'll see if we can give a try

Best Regards
Bo