As long as there’s been development, there’s been testing; from the early days of waterfall, to figuring out how to overcome the hurdle of agile, and establishing where testing fits amid the continuous development loop. Now, DevOps has created a similar hurdle – one we are hoping you will have a better understanding of as we use this article to help you navigate this new world.
According to Amazon Web Services (AWS), DevOps
“…is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes.”
This definition is a common one and while I can’t disagree with it, there is one critical missing piece. It should read “… to deliver high-quality applications and services …”. It may be implied that all delivery should be of high quality, but over my career, I have seen far too many examples of quality being sacrificed for the sake of meeting a deadline. It’s not that the teams didn’t care about quality, but the timeline is usually the driver, so the term “good enough” becomes more common than it should be. Maybe it was good enough, and maybe it wasn’t, but to really know and make that decision, we need testing to gather the information about quality.
Deadlines are important, there is no doubt, but testing takes time. Is that why the name and definition of DevOps leaves out testing or quality? I’ll leave that as food for thought.
Testing at high velocity
If the goal is delivery at high velocity, then testing needs to take a deeper look at how it accommodates the delivery method. After all, DevOps (or the next iteration thereof) is here to stay and there is too much at stake for poor quality.
When looking at applications and how to deliver quality in this high-paced model, a quality plan needs to be built early – preferably at the beginning – in the development cycle. The plan should lay out how testing will be performed throughout, including once the application has been moved to production.
Getting to the release of an application is really just the beginning. Features will change, deficiencies will be found, how users engage with it will change; these are all expected and required for businesses to adapt quickly.
Testing teams need to have many tools in their toolbox, including a deep understanding of how software is developed, to assess the critical testing areas. They must, as well, understand the value of automation to efficiently validate the quality of existing functions.
Test automation meet DevOps
Automation is not new to the DevOps world. In fact, its very nature means that it is already well positioned to fit into the continuous delivery model. Historically, testing was left to the end of a delivery cycle, but times have changed. It’s now understood that leaving it to the end increases the likelihood of failure, and increases the risk of releasing poor quality code.
Similar to agile, test automation should be created to deliver small, frequent updates that allow testers on your team to focus on new functionality with confidence, knowing that the test automation suite is there to provide the necessary checks on existing functionality.
Automation is not simply the automation of tests, but also the automation of the process, which then allows the test suites to be executed regularly as part of a continuous integration (CI) and continuous deployment (CD) pipeline. Notice the wording; testing and quality are not explicitly defined as part of the pipeline. I won’t suggest that we change the name of CI/CD to be something else, but it’s on all of us to ensure that continuous testing is as much a part of the conversation when speaking of the CI/CD pipeline to ensure a consistent, high-quality product.
DevOps’ benefit to the development team is obvious; it provides immediate feedback on the quality of the code they just created. This is every bit as important to the test team because they’re getting immediate feedback on the effectiveness of their test automation suite, which allows changes to be made before the suite becomes out-of-date and unmaintainable.
The challenge of change
It’s not difficult for testing and quality to fit into DevOps, but it does require a commitment at all levels of the organization to be successful. Excuses for testing not being a key component are easy – not enough time, not enough budget, and the list goes on. Making it work will require dedication and change, which can be difficult and may not result in an immediate reward. What is known is that when it is done right, all members of the organization will sleep better knowing that they not only delivered code at a high velocity, but also at a high quality.
There are a few key items that should be part of any move to higher quality software in the world of DevOps:
- Commitment
- Early planning
- Understanding risks
- Quick and iterative test automation
All of this sounds simple and, with the right commitment, it can be. Our Automation Services team at PQA is here if you’d like to have a conversation about testing and quality in DevOps to help you and your team deliver high-quality software for your business. Reach out, and let’s talk testing!
Jonathan Duncan is the VP of Customer Success at PLATO Testing based in Fredericton, NB. Jonathan has over two decades of wide-ranging experience across all facets of the software development cycle. He has experience in a variety of industries that stretch from the public sector to start-ups in satellite communications and everything in between. Having worked in organizations from both the development and testing standpoints provides Jonathan with the ability to see problems from all aspects allowing for complete solutions to be delivered.