Today, mabl revealed our 2022 Testing in DevOps Report, a pulse check on how software development organizations are adopting DevOps and quality engineering on the path to digital transformation. Since launching in 2019, the Testing in DevOps Report has delivered a quality and testing-centered view into how organizations are connecting agile development practices with critical business outcomes like customer happiness, faster deployments, and smoother releases.
This year, over 500 software development and testing professionals shared insights into the state of DevOps, testing, and software development. They reported that delivery cycles are indeed accelerating - but also by how well an organization has adopted automation, particularly test automation.
Like last year, we found that DevOps challenges are uniquely holistic, ranging from cultural issues to a lack of leadership and skills to successfully navigate the transition. Budgetary issues and technology limitations also play a role, but we continue to see process and people challenges as the most significant obstacles.
Though the slow process of change was the leading DevOps challenge, no issue held an outright majority, indicating that these obstacles are intertwined. As intimidating as solving a group of issues seems, the 2022 Testing in DevOps Report reveals that many organizations are successfully reaching full DevOps adoption by holistically automating their development pipelines, engaging in collaborative testing practices, and accelerating deployments.
A major trend identified in this year’s report was that software development organizations are moving faster. 75% of respondents reported increasing their overall deployment frequency in the past year. How significant those improvements were, however, was closely tied to DevOps maturity.
Teams with a high degree of DevOps maturity were more than 5x as likely to deploy multiple times per day compared to aspiring DevOps teams, with no fully DevOps organizations reporting annual or bi-annual deployments.
When looking at the factors that differentiated these fast-moving teams, it became clear that mature DevOps organizations evolved their pipelines and their deployment practices in tandem. High-velocity DevOps teams were also significantly more likely to have highly automated pipelines.
Though the majority of survey respondents (72%) said that technology limitations weren't inhibiting DevOps adoption, the clear connection between overall pipeline automation and deployment frequency are a signal about the future of tools seeking to enable DevOps.
The tools DevOps teams use need to be more than functional; they need to be easy-to-use and integrated across development pipelines in order to support better processes. 72% of respondents said that technology limitations aren’t holding them back on the road to DevOps, indicating that functionality is less of a concern than having tools that make it easier to collaborate and build at a high velocity.
Not only are mature DevOps teams with automated pipelines deploying more frequently, they’re doing so with confidence - and their customers are noticing. Unlike the old adage of “move fast and break things,” successful software development organizations are the most likely to have expansive testing practices and cultures of quality.
Teams with fully automated pipelines were not just the most likely to have accelerated deployment velocity, they were 9x more likely to rate test coverage as good/excellent. Yet the increase in deployments didn’t increase the number of bugs in production. On the contrary, teams who were able to innovate faster had much happier customers.
Teams with fully automated pipelines are 2.5x more likely to rate their customer happiness as good or amazing. What this tells us is that no facet of DevOps - automation, collaboration, and acceleration - happens in a vacuum. Organizations who can evolve their toolset, processes, and people continue to be DevOps leaders and those best positioned to retain customers in an increasingly competitive digital economy.
We’re so excited to share the full 2022 Testing in DevOps Report; I encourage anyone invested in software quality to read our findings. As mabl’s co-founder, I’m excited (but not surprised) to see the important role software testers and quality professionals are playing in DevOps success.
When quality is elevated, the entire software ecosystem feels the impact. Deployments happen more frequently, with less stress, and fewer bugs. Developers can build new products with more confidence, quality teams are empowered to improve the user experience, and consumers benefit from delightful, inclusive digital experiences that make daily life easier. Join us on Wednesday, December 14th for an engaging discussion to discuss the full report and its implications for DevOps and quality engineering adoption.