Software Testing in the Cloud Era

As we continue into the era of hyperscale Cloud computing and high-frequency DevOps we can examine the increased role and importance of QA and software testing. 

A recent research report from Canonical highlights how we are still in the very early days of the shift to 'Cloud Native' computing. Pioneered by Netflix, this is an approach to development based on the use of containers, Continuous Delivery and Microservices software architecture, and represents the most significant evolution of enterprise IT that will impact software testing.

Software Testing and Kubernetes

For starters, we can look at Kubernetes and its relationship to testing. Kubernetes is an open-source system for automating deployment, scaling, and management of containerised applications that has become the centrepiece of the Cloud Native movement.

Sauce Labs explores this in What Does Kubernetes Mean for Software Testing.

First, they highlight that the main functionality of a containerised application isn't different from a non-containerised one, so the functional testing will remain the same. The main evolution is the ability to leverage the environment itself to support the activities of testing.

Packaging an application doesn’t change how these scripts run, but if you have a set of automated tests built, they can be incorporated into the continuous build pipeline used to produce your containers so tests can be performed and reported on with more consistency and much more timely results.

Writing for TechBeacon Glenn Buckholz highlights how Kubernetes enables complete application deployment models to be defined in YAML, enabling the environment to be easily replicated for testing purposes, bringing critical benefits to testers including a decreased dependence on shared environments, direct access to logs, the ability to get to a complicated failure state quickly and cheap disposable environments.

In short, it empowers testers with the same dynamic Cloud-powered capabilities the developers enjoy and enables both to unify their work around the same consistent environment.

Cloud Scale Testing

Then we can consider a broader context of ‘Cloud Scale Testing’, which can be defined to have two main dimensions: The testing of whole infrastructure operating at a global Cloud scale, and the use of Cloud to implement testing practices at an equivalent scale.

Netflix offers a great tutorial on the first part, in their 2016 presentation Microservices at Netflix Scale. 

What this demonstrates is that testing practices can be applied to the whole IT environment, not just isolated application code, and the key to this is primarily the assumption that failures will occur and therefore the system is engineered with a capacity to rapidly adapt to them. Netflix pioneered and operates an approach of ‘circuit breakers‘, implemented through their Hystrix component.

Again automation of these adaptations is critical as is the ongoing destructive testing of the environment to ensure the capability. They implement these tests at a very large scale, simulating the failure of entire AWS regions, demonstrated in visual form at 35m:35s.

Speaking at Cloud Next the founders of Mabl provide an in-depth review of the second aspect - Harnessing the Cloud for enhanced testing capabilities. They talk through deployment across the Google Cloud and how this enables large scale ‘intelligent testing’ features.

Primarily this intelligence refers to adaptability and automation of high volumes of testing functions. For example, testing can adapt to changed forms and code so it doesn’t break when these changes are made, and data analysis is performed to surface insights for testers and developers that eliminates much repetitive grunt work for them. All of this can be integrated into the CI/CD workflow so that it happens automatically as part of the build process.

At the start of the talk Dan Belcher sets the scene for why these new capabilities are needed:

“Development cycles are getting shorter and new features are coming faster than ever… There simply isn’t enough time to test”. The old ways that we had thought about QA are simply not holding up anymore due to the continuous development and delivery. More and more releases are coming our way and the challenge to us as an industry is how can we achieve that velocity while still being able to deliver high-quality products and build our user’s trust.

His presentation states the key improvements needed for software testing in the Cloud era, being able to deliver tests that: 

• Adapt seamlessly to applications that change rapidly. 
• Will run in the cloud with exceptional scale and speed. 
• Produce insights based on the output that is actionable. 
• Integrate with automated CI/CD pipeline. 

In conclusion, we can see that rather than decreasing the importance of testing, the shift to Cloud Native computing magnifies its importance even more so. Development organisations are creating much larger, much more complex and dynamic code estates than ever before, and being able to assure its performance and reliability is therefore equally important as ever before.

Modernising testing practices to also operate at the same Cloud scale is therefore the fundamental challenge and requirement for development teams in this new era, a journey that 2i can assist with.