Let the Shell be Your Pal

In his talk at STPCon in April, Simon Stewart, the initial developer of Selenium WebDriver, discussed a variety of bad practices related to using WebDriver and things that he wished people wouldn’t do. Sure, there are many excellent uses for GUI automation tools, and driving the behavior of an application with GUI automation can be a good thing. Often, though, we end up doing things that can be counterproductive or just plain taxing to the system in ways that are not useful.

As I considered a number of the challenges being described, one thought kept popping into my head: Why aren’t more people taking advantage of the basic shell commands provided by their systems?

What is the shell?

If you have ever interacted with the Command Prompt window in Windows or the Terminal in MacOS or Linux, then you have interacted with the shell. There are a variety of shell versions — PowerShell is what’s used on Windows, and many shells are available in both MacOS and Linux. The most popular today is the Bash shell (which stands for “Bourne Again Shell,” as it contains numerous improvements on the earlier Bourne shell that was created back in the 1980s).

The shell is a direct way to interact with software programs, utilities and built-in options unique to the shell environment. What’s more, shells offer many of the standard programming options that a variety of scripting and compiled programming languages offer.

How to leverage the shell

Perhaps the most basic and, arguably, important process is the setup of test environments. The majority of these setup and configuration commands will not be run from a web browser, especially not in the early stages of bringing up a machine to get it ready for testing. For these options, interacting with the shell is imperative.

The shell allows a great deal of power over how the environment gets set up. Users can string together multiple commands in a sequence. It may seem obvious, but if you find yourself typing a variety of commands to set up an environment, you can just as easily put those commands into a script file, make that file executable, and run the script to handle all the tasks in one go.

For many, that is plenty, but there are lots of additional options available to the tester willing to spend just a little more time getting to understand how to program in the shell.

Setting parameters for different setups

The shell allows for conditions, looping and other options to execute commands in a variety of ways. It’s possible to set up case statements that will contain configuration information for several machines and to pass on only the information relevant to the particular machine I am setting up.

As an example, I have several machines in Amazon AWS that I use for active testing. Depending on which machine I want to test with, I have a variety of associated environments and variables. By creating case statements in my shell scripts that contain the values necessary for those machines, I can make a command line parameter calling out that particular machine. By doing so, my script knows to reference just the values relevant to that machine and to copy them over at the appropriate time.

By setting variable values and making conditional statements, I can also create SSH tunnels to run commands on peripheral machines at key times, to help make the necessary processes available to each system under test.

Querying databases and storing values

One of the more useful tools for a variety of my tests is the psql command-line program, a client for PostgreSQL databases, to find certain values. I can SSH to a particular machine, run psql, and issue database queries to get table outputs or view individual values. Often, those individual values are important to the work I’m doing, and having a way to pull those values in while I am setting up my tests is vital to my success.

Psql can be called from within a shell, and the output of such commands can likewise be stored as variables in the shell. From there, I can use those variable values to determine such things as active users of a system, who has credentials and who doesn’t, and to make assignments based on those values.

Adding users to a system

I remember Simon at his most frustrated when he described seeing people using WebDriver and other tools to go in and create users. Yes, being able to create a user is a valid test, and it may well be an important test, but it is important once — maybe a few times, if there are a variety of paths to adding users to an application or a system. But to do it for every user, especially if tests require dozens or hundreds of them, is a waste of time that could be used performing more important tests.

To this end, again, a simple “for” or “while” loop can be used to iterate over a CSV file and add users one by one, if that is desired, or through a bulk transaction with a database query.

An ever-growing toolkit

The shell is not going to be the best fit for everything, but a great deal of my testing, especially setup and teardown processes, are done with shell scripts. What’s more, these scripts are easily portable and can often be used with any bare-bones system.

Simply by copying over a tar file, unpacking it and running scripts, many tasks can be handled in an orderly manner with little to no need for intervention on my part. Additionally, whenever a new option is realized, I try to find a way to pull that into my growing suite of shell commands.

In many cases, CI and CD environments are driven by shell scripts, particularly moving repository files up to release servers and then pulling down those repos to perform installation steps.

Interacting with APIs

By using commands such as “wget” and “cURL,” it is possible to interact entirely with a system using nothing but shell commands and, based on the responses received, to determine the fitness of the system and the appropriateness of the responses. This can supplement UI tests, giving me additional time to examine other aspects and behaviors of my system.

It’s not all rainbows

To be fair, there are a number of issues anyone who uses shell scripts will face.

Many programming languages have much more robust systems of data types and variables to work with, and, in certain cases, a dedicated scripting or programming languages makes much more sense than trying to make a shell script do everything. While shell scripts allow me to create functions, those functions don’t take parameters the way other programming languages do. That means processes that would be much cleaner and more straightforward in a language like Ruby, Python or Java would take more steps — in some cases, many more — to meet similar objectives.

Shell scripts can also cause irreparable damage if run at the wrong time or in the wrong place, especially if run with root privileges. Also, while the shell itself has flow control, looping and other testing options that are helpful, nearly every command issued comes from the operating system or a utility within the operating system, each of which has its own rules about how it handles input and output of data, parameters being passed to it, and return states that vary depending on which utility is being called. This is especially the case with Linux, where many commands depend on piping, redirecting and otherwise capturing the output of one command to drive the input of another, sometimes in long chains of interactions.

Use shell scripts when they make sense

There are a great many areas where GUI testing or using a testing tool will be vital to success with a project. But what I am encouraging is to look for places where there may be bottlenecks in automated tests and to examine why those bottlenecks are there. Often, that process shows a process or task that is not well suited for a UI test and can be handled in a better way by a different tool.

When performing tests, I find that leveraging the shell complements the UI tests I do and allows me to use the UI tests I create for the areas that actually need that interaction. I can often free up significant time and resources with a battery of small, targeted shell scripts to help take care of things that my UI tests are less well equipped to handle.