8 Tips for Testing in a Containerized Landscape

Containerization generally takes place with considerable, and sometimes total, automation of testing. Requirements will be expressed in terms of production software — that is, how the containers operate when running on production-level container hosts. It’s good to have automated testing of those requirements.

1. Protect your style

Don’t let those be your only tests, though. Suppose you have an ambitious application packaged in containers, with continuous integration (CI) that ensures that each commit from a developer triggers construction and testing of production-ready containers. Even before those heavy steps start, though, your CI can launch several lightweight steps: It might run a human-language spellcheck on diagnostic texts, or check that source code all conform to the organization’s style guide. If those sorts of tests fail, it’s best to return the results to the coder in milliseconds.

The alternative, where all building is done and then testing starts, is just bad ergonomics. Quick responses, to the extent that they’re accurate, are best for maintaining a programmer’s “flow”.

Some tests take intolerably long so that programmers can’t wait for their results. Don’t abandon those tests; just schedule them independently of what programmers do.

2. Plan your continuous testing

Don’t just accept finished containers as the outputs of CI that are tested in isolation from the CI that produced them. Work with your programmers to plan your CT.

As the previous tip mentioned, the “raw” source code can be scanned for some tests; depending on the languages and other technologies involved, other tests might happen at compile-time, package time or any of the stages of a multistage container build. In general, it’s best to test an artefact as early, or as far to the “left,” as possible. When a revision has errors, deliver those results to developers sooner rather than later.

3. Containerize testing

Containers can be not only objects of testing, but also subjects. Most of the tips in this article are about how to test containers. If your organization is taking advantage of container technologies, though, those same technologies are available to improve testing itself.

Do some of your tests require special configuration or installation of hosts? Wrap up those configurations in containers and have them instantly available to run everywhere, from programmers’ desktops to a data center, in the middle of debugging an incident.

4. Counteract vulnerabilities

Containerization solves security problems, right? Wrong. While containerization certainly moves security vulnerabilities around, and in many cases narrows them, security risks never vanish. Containerization is good for security (and testing more generally!) when it comes to minimizing differences between development, staging and production environments. Differences remain, though.

In any case, you’ll want to develop expertise in the vulnerabilities of containerization and how to mitigate them. Are the containers you test dependent on containers from public container registries? Quite likely. Are public container registries safe? No — but then again, the safety of in-house constructions isn’t absolute either.

Learn how to verify images, control networking tightly, and what it takes to ensure your particular container technology doesn’t leak information to container hosts.

5. Test microservices

If your move to containerization coincides with reliance on microservices, you’ll need to learn the peculiarities of testing microservices, especially for security.

6. Modernize your policies

Organizations live through their policies. Even the most relaxed-looking organizations are likely to have cultural expectations about what policies not to have. Smart testing respects and leverages organizational policies. A company that takes its SOC 2 compliance seriously, for instance, doubtless has complementary testing already in place.

Containers are a disruptive technology, though. While existing policies might be entirely rigorous for older technologies, they’re unlikely to account for needs that didn’t exist at the time they were composed. One common example: While it’s important to firewall Kubernetes controllers with care, formal policies might not recognize the centrality of Kubernetes at all, and utterly neglect this requirement.

Strong policy cultures often fail to grasp that technology like containerization can still slip through the cracks and leave the organization vulnerable to particularly large security hazards. The remedy is to refer back to the original goals policies were crafted to serve, work with security and compliance specialists to update the policies, and figure out how to apply them sensibly.

7. Test at scale

Another challenge containerization doesn’t address directly is testing at scale. Containerization can indeed enable scalability because when done right, containers are great for handling loads that multiply.

To test that they’ve been done right, though, is a separate and far-from-obvious effort. Testing “in the large” is a specialized expertise. The remedy should be familiar by now: Identify the need, and plan its solution with the help of the whole team.

8. Unify the vision

Done well, a migration to containers can unify and align several separate efforts. Containers make automated testing more feasible, and CI helps fulfil compliance requirements, and consistency through the software development lifecycle promotes DevOps agility, and container predictability eases testing in production, and … With the right attitude and planning, containerization can bring your whole team together, so everyone from sales to development sees a payoff.