Five Techniques for Continuous Integration

The brave new world of continuous integration and continuous deployment is exciting: It promises, among other benefits, better and more consistent testing. Too often, though, the reality is far different. Despite our best intentions for better control over errors, our CI and CD practices introduce new sources for errors.

A little care with crucial details eliminates many of these problems. Your goal should be to have your continuous integration tool — for this article, I’ll talk about Jenkins, but it also could be Bamboo, TravisCI or another alternative — work for you rather than your having to work around it.

Here are five ideas to help ensure that outcome.

Test the Pipeline, Too

Left to themselves, developers are likely to think of “testing” narrowly, and the source code they write as the central asset of the organization. A testing department has the opportunity to communicate that the best results come when all the parts of a solution team up in a balanced way.

In this perspective, testing’s target extends beyond source code; it also includes other elements such as third-party libraries, the repositories used, documentation, and the automation of Jenkins and other tools. It is crucial, in fact, to validate all Jenkins configurations. That means not just verifying that they produce good applications or installations when the repository is healthy, but also that they respond appropriately to errors.

Consider this even more concretely: a typical test of a product might be something like: “Make sure that a login screen comes up within two seconds after the application is launched.”

To test a Jenkins configuration likely involves items such as: “If a source file in the repository includes a syntactic error, then a notice goes to $RECIPIENT within $CI_CYCLE minutes.”

Just as in other kinds of testing, it is important to test Jenkins both in its happy (successful) and unhappy (failure) branches.

Make sure that your tool is under control

Source code must have a home in an authoritative repository. While developers assume that to be true about what they write, that same repository is where your Jenkins configuration (all config.xml-s and *.txt-s, to start) also needs to live.

At the same time, the base Jenkins installation deserves to be documented. To have both base Jenkins and its site-specific customizations documented and controlled should give reproducible results.

Everyone who works in business continuity or disaster recovery knows, however, that backups only have value when recovery is regularly tested. This is another test that deserves periodic execution: ensuring the Jenkins installation can be recreated as though the local Jenkins server had been lost.

Jenkins creator Kohsuke Kawaguchi recognizes that Jenkins remains brittle. The value Jenkins provides is too great to rely on it without good backups and tests of recovery from those backups.

Minimize script complexity

A fully-automated, tested, backed-up Jenkins pipeline is an important milestone on the way to your continuous testing or continuous deployment goals. It is the base on which an organization begins to build real value. Once that base is in place, decide deliberately how much configuration to maintain through the web (TTW). Jenkins and similar products make it easy to edit and define simple pipelines in the Build tab.

In principle, those definitions can expand to handle more complex cases; the content of the Build tab can be arbitrarily long and involved. However, for most organizations, a better strategy is to restrict Build to something as simple as

/opt/local/$PRODUCT1/build.sh

or

powershell /special/build1.ps1 automatic

In this scheme, build.sh embeds all the construction logic necessary for $PRODUCT1; most crucially, build.sh can be tested, measured and versioned outside Jenkins. Reserve Jenkins for high-level automation of business operations, and keep the implementation complexity in scripts maintained independently of Jenkins.

Recognize the “jenkins” user

A conventional Jenkins installation executes processes with the jenkins user as owner. Depending on the details of your operating system and other services involved, this means system administrators might need to:

• Add the jenkins user to the docker group with usermod -aG docker jenkins
• Run triggers within Jenkins crontab
• Allow anonymous read access so that developers and testers can view job results

Security settings for Jenkins are a large and complicated subject, as is inevitable for an architecture that supports plugins and scripts; it can be nearly a full-time job to keep track of In-Process Script Approval for a highly customized installation. The first step to a comprehension of Jenkins security is awareness of the jenkins user.

Restart wisely

Jenkins’ restart behavior is subtly different in different releases. In general, Jenkins distinguishes “restart” from “safe restart”; the latter finishes running jobs before restarting.

One common way to restart safely is TTW at <URL: $BASE_URL/safeRestart > or the corresponding Jenkins command-line invocation:

java -jar jenkins-cli.jar -s $BASE_URL restart

There are also command-line variants of the safe restart that depend on the host operating system, such as the following for macOS:

sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist

Conclusion

Embarking on continuous integration and continuous deployment without also employing continuous testing will only introduce more sources for errors. A CI tool such as Jenkins can help you, as long as you are careful with the details.

Precisely because the tool is so powerful, it&#39;s essential to understand and test Jenkins pipelines with the same care as all the other software components that go into a finished product. By using these five techniques, you can get the most out of your CI tool.