5 Challenges of Building Automated Tests

The software testing field has seen a lot of advancements in the past decade. It has evolved from being completely manual to its current state, where automated testing is widely used in conjunction with manual testing to make testing more efficient and get faster feedback on the application under test.

But not all companies have followed this transition path. There are various challenges that continue to be an obstacle for organizations, from new startups to large companies.

What are the challenges?

On a high level, there are five challenges teams face when building automated tests. Focusing on these key factors will help you build faster, more efficient, more stable automated tests.

1. Skillset

We live in a competitive market where the need for skilled resources often outweighs availability. Organizations invest considerable time and effort in finding skilled testers, interviewing them and onboarding them, all in the hopes that they will build a robust automation framework to catch defects faster.

While some companies have the money to do this, a lot of startups and small companies do not have sufficient resources to attract skilled testers. In the end, the existing business representatives, developers, or stakeholders end up testing the applications themselves manually. As a result, release cycles get affected as testing becomes a bottleneck for releasing the product on time, and the company is unable to meet growing customer demands.

2. Authoring

Once a testing team has skilled testers in place, the next challenge is to author the automated tests. Testers are often eager to start building the automation framework and quickly write multiple tests. But in that process, they sometimes fail to pay attention to important aspects of creating a durable automation framework.

These aspects include having reusable components, doing data-driven and keyword-driven testing, using parameterization and having effective waits. When these factors are overlooked, it leads to teams building highly coupled and low-cohesion automation suites that do not meet stakeholder expectations.

3. Initialization

While building automated tests, attention needs to be given to the initial states of the application.

For example, say we are automating different flows on an online shopping website, and testers write an automated test to add one item to the shopping cart. The first time the test is run, the test would add one item to the cart. What happens when you run the same test again? If we do not pay attention to reset the application to its initial state, the test would add another item to the cart, and now instead of having one item, we will have two items in the cart. When the same test is run multiple times, the cart size will increase, leading to unexpected behavior.

Not paying attention to the initial states of the application causes a lot of grief to teams and affects release schedules.

4. Maintenance

Maintenance has been the biggest challenge with test automation for the past several decades. Let’s say we write 10 tests and they all pass. The initial results make teams happy, but then they find out the next day that most of the tests are now failing due to factors such as the test environment going down, the application crashing and developers making changes to the application.

It is typically recommended that about 30% of a tester’s time be allocated for maintaining tests. Imagine how much more productive we could be if even 5% of the time could be saved? There are many solutions and tools to address the problem of maintenance in tests, saving testers and companies considerable time, cost and effort.

5. Scaling

As the automated test suite grows in size, it becomes more important to manage it efficiently. If we have thousands of tests in the suite and it takes about four hours to execute and get feedback from the tests, what happens when 200 of the tests fail? We would need to inspect all these failures, figure out what is expected and what’s not, and identify flaky tests that need to be stabilized. This consumes a considerable amount of time and pulls teams away from working on higher-priority work.

The challenges with scale apply to hardware resources as well. When there are only a few tests to run, it may take minimal hardware resources, but what happens when there are thousands of tests to run? A lot more server and computing space would be necessary. This is the reason many companies get subscriptions to cloud services, to save cost and time on procuring hardware resources.

Teams need to run more tests in parallel and find problems as quickly as possible. For this to happen, the necessary hardware resources need to be in place.

Overcoming These Challenges

The need for automated tests is only increasing, and teams are bound to face at least one of these challenges at some phase of the software development lifecycle. But these problems can be minimized if corrective actions are taken at each step of the automation framework building process.

First, there needs to be increased communication and collaboration between teams to ensure everyone understands the goals and objectives of building a test automation framework. Start by automating a small set of higher-priority tests, ensure they run consistently over a period of time, and then build on top of the existing stable tests. Create a future vision of how the framework could be scaled to support other initiatives and made more flexible, so there’s potential for reuse with less rework.