Deciphering what comes first, quality engineering or DevOps, is a bit like asking if the chicken preceded the egg. Everyone has an opinion, but no one really has a definitive answer. The truth is most likely somewhere in the middle, as many software development companies are in the process of adopting both DevOps and quality engineering. The two practices are deeply intertwined and even overlap.
First, let’s establish our core definitions:
- DevOps is both a philosophy and set of practices that seeks to accelerate product velocity through improved collaboration, increased automation, and incremental, data-driven improvements. By eliminating the traditional siloed approach to software development, DevOps enables teams of all sizes to simultaneously improve their product and deliver new code faster.
- Quality engineering is a discipline that incorporates best quality assurance practices and data-driven testing (both manual and automated) into the end-to-end customer experience and drives organizational growth.
There are a few common themes. One, both DevOps and QE rely on data and automation to improve the development process as well as the application or website itself. Second, each organizational approach emphasizes collaboration, especially across different teams involved in the software development lifecycle. Test automation meets all of these demands, making it an essential component in the DevOps and QE adoption processes.
Scaling Test Automation in DevOps
Thanks to its central role in DevOps adoption and quality engineering, an effective test automation strategy is one that can easily scale across the DevOps pipeline. This includes growing the number of people participating in testing as well as expanding the variety of automated tests being performed.
Managing a Culture of Quality with Test Automation
The most successful DevOps teams - those that see improvements in product velocity as well as product quality - are differentiated by their cultures of quality.
As illustrated by the chart above, which comes from mabl’s 2021 State of Testing in DevOps Report, the most significant change across DevOps adoption stages is the number of respondents that said “everyone” is involved in testing. Fully DevOps teams are the only ones more likely to say that everyone is engaged in testing, rather than just quality teams.
The benefits of this are obvious: more testing, more bugs caught earlier in production. But having more people contributing to testing can quickly overwhelm any sense of order, making it hard to ensure that testing activities align with overarching quality goals.
With the right test automation framework designed for DevOps, scalability across the entire team is built right into the platform. Automated testing should be manageable, transparent, and collaborative for effective test management in DevOps. Features like an Activity Feed help QE teams understand all testing activity taking place in a particular workspace, with identifiers for individual users so that it’s easy to support a large number of testers - even if they aren’t QA. At-a-glance test management helps QA teams identify where the transition to quality engineering needs extra support, track quality goals against testing activities, and support testing collaboration.
Designing An Efficient Testing Strategy
Automation is central in DevOps because it helps people be as efficient as possible, enabling faster workflows without sacrificing quality. Test automation is no different. As software testing becomes more complex through the increasing use of APIs, the expanding definition of end-to-end testing, and the need to align testing with the customer journey, quality engineering teams are turning to automated testing for an efficient quality strategy.
Looking again at the mabl State of Testing in DevOps Report, most teams that use test automation are only focused on UI/Functional tests, UI tests, or regression testing. Most test automation platforms are fairly good at performing these types of tests, even if some require extensive maintenance to keep up with changes to the application under test. But as DevOps demands quality teams test more complex software in less time, limiting an automated testing strategy to just the top three types will ultimately slow the entire team down and hurt product quality. Consider that API usage is increasing across all industries, or that 57% of consumers won’t recommend a business that provides a poor mobile web experience: ignoring API testing or cross browser testing could have a real impact on the success of business.
As testing strategies evolve with product complexity and the user experience, quality engineering teams need centralized test automation platforms that allow them to monitor all testing taking place. Simply adding new test automation tools risks a disconnected and siloed testing strategy that wastes time and resources. With all test automation capabilities housed within a single platform, quality leaders can continuously evaluate their automated testing strategy for maximum impact, testers (QE and non-QE alike) are able to more easily execute new types of tests, and testing can adapt at the pace of DevOps.
Test Management is Quality in DevOps Management
As an organization’s DevOps practice matures, quality engineering leaders need to be prepared to pivot their testing strategy on a dime. Effective test management not only makes it easier to start collaborating with other teams in the organization, but also enables more rapid improvements in quality strategy. This means that testing reflects both the real user experience, the product as it evolves, and overall DevOps adoption. Centralizing automated testing in a single platform allows quality leaders to build a culture of quality by scaling testing across the DevOps pipeline and optimizing the types of tests being executed for maximum efficiency.
Help us understand the changing connection between DevOps, quality engineering, and software testing by taking our 8 minute survey for the 2022 Testing in DevOps Report!
