Launch Your Continuous Testing by Making the Most of GitHub

There are several reasons you may decide to begin continuous testing: earlier opportunities for QA, increased risk mitigation, and the ability to keep up with faster dev sprints. And if GitHub also hosts projects you’re working on, you’re in luck! Plenty of others have passed this way before, and you’ll find the path forward is relatively smooth and well-lit. But a few preliminaries will help your voyage.

Tooling up

First, much of the existing literature refers to CI (continuous integration) or even CD (continuous delivery or continuous deployment) rather than CT, or continuous testing. Even if you know your focus is specifically on CT rather than the broader span of business aspects that CI and CD cover, be prepared to read up on CI to get the information you’re after.

Don’t be distracted by how often release candidates are generated, or disputes about elaborate rules for administration of deployments. Your attention is just on the CT part: You want to make sure each commit launches its own regression test.

A second cultural point to make is that while tooling is important, CT tools are rather convergent in their basic capabilities. When starting out with CT, it’s generally more valuable to make a reasonable choice swiftly than to try to decode the abstract technical capabilities of different competitors in relation to your project’s specific requirements. Anything you can do to comply with existing organizational standards or to leverage the experience of your team is more likely to pay off than an exhaustive search for technological compatibility.

If your organization already has licensed CI from a particular vendor, use that. If you happen to belong to a great user group where you can learn from experts with one specific tool, use that. In any case, choose a CT tool, make the most of it, and begin to enjoy the benefits of CT. If, some day in the indefinite future, you find that your specific choice has a limit you can’t work around, much of your CT scripting or configuration will still apply within alternative frameworks.

A different way to say the same thing: Most of the better CT tools allow for integration with GitHub and common CI processes. Expect that you’ll advance rapidly once you start your CT work.

CircleCI example

CircleCI is one of several frameworks that make initial CT experiments easy. If you have an existing GitHub project, CircleCI lets you sign up with those credentials.

It’s crucial in your first quick CT configuration to start small. CircleCI and competing services are mind-bogglingly flexible. Don’t try to understand everything they can achieve; focus instead on the simplest possible configuration that applies to you.

Refer to this demonstration site to confirm your steps.

The recipe

Add a new file .circleci/config.yml with content
version: 2.1

jobs:
build:
docker:
- image: circleci/php:cli

steps:
- checkout
- run: echo '<?php print "CT success."; ?>' | php

to your repository. Are you not a PHP programmer? That’s OK; once you finish this first exercise, it’ll be quick work to change to a better fit. The aim for now, though, is to work with something as concrete as possible, so you can start to experience CT for yourself.

With config.yml in place, visit CircleCI, and log in using GitHub — you’ll see a button labeled exactly “Log In with GitHub.” That takes you to your dashboard, where you’ll need to follow your project. As soon as you follow your project, your dashboard should show your project next to a green “SUCCESS” label.

That’s it! As the label puts it, you’re a SUCCESS.

What to do next

While that first step is the most important, it is only the beginning. Next you’ll want to view the log of your SUCCESS, which should finish with the result

...
CT Success.

Experience the value of CT by introducing an error into the configuration; you might, for instance, change the run line to

- run: echo '<?php printx "CT success."; ?>' | php

Soon after you commit this to your GitHub repository, your CircleCI dashboard will light up red with the complaint “… syntax error …”

Do you see the value? As you add real tests to your configuration, they’ll automatically run against every commit. Never again should it be a surprise for an error to sneak into a branch, because every commit will be tested.

Next up

You’ll make rapid progress from this point:

• Swap out PHP for a language environment that fits your own needs better, or even customize a Docker image for your project
• Introduce tests you’ve already written into your configuration
• Define rules for specialized testing scenarios or reports on who gets what test results
• Read up on jobs, workflows and orbs so you make the most of the freely available artifacts other teams share publicly

Keep in mind that essentially everything in your configuration is variable, and even variably variable. Start small, get something working, and keep improving it in the direction of your vision of what CT should be for your team.

Minimize your configuration — that is, while your CT configuration language has enough power to do almost anything, you’ll generally be better served to script tests and filters outside the CT, and just use the CT to “glue” together working parts. With a little practice, powerful testing tools integrate nicely in nearly all CT schemes.

Right now, wherever you’re starting, you’re just a few minutes away from dramatic CT progress.