2021
When we left Google to start our next company, we spent months interviewing hundreds of software teams about the most significant challenges that they face. We weren’t surprised to find that most teams were focused on accelerating innovation by embracing agile and DevOps. But we were shocked by the number of teams that pointed to ineffective quality assurance as their primary obstacle. In fact, among our first 100 interviews, none reported that they were satisfied with their QA efforts.
Over 600 professionals responded to our survey, with a broad range of quality and software development roles represented. The majority of respondents were those on the front lines of quality, including manual testers and quality analysts (30%), quality engineers or software developers in test (20%), or quality managers and directors (13%). The rest of respondents held various roles in product development, software engineering, and leadership positions.
Survey respondents also worked in a variety of professional environments, giving us insight into how quality strategy and DevOps adoption are evolving in organizations of all sizes. The plurality (38%) reported working in large enterprises with over 1000 employees. Another sizable contingent (26%) was employed in small businesses or startups with just one to 100 people. Professionals at midsize organizations with 101 to 500 employees represented 19% of respondents, while those at medium enterprises with 501 to 1000 employees made up 17% of survey responses.
To set the baseline in DevOps trends, we asked respondents where they are in DevOps adoption. Adoption was relatively evenly distributed, with 28% of participants indicating that they’re making strides towards DevOps adoption. One-in-four participants reported aspiring to adopt DevOps, but still relying on manual deployments to put updates into production.
Unsurprisingly, significant roadblocks remain in the way of full DevOps adoption with completely automated development pipelines. But as the democratization of DevOps adoption indicates, technology is no longer the biggest challenge: a whopping 82% of survey respondents named non-technological issues as the main inhibitor to DevOps.
.png)
Though accelerated product velocity is the most commonly cited benefit of DevOps adoption, our research showed the speed of development remained relatively flat in 2020-2021. Though a third of teams are now deploying new code at least once per week, almost half still deploy less than once per month. The number of teams that deploy new code daily or multiple times a day (15%) is roughly equivalent to those deploying on a quarterly basis (16%).
The data also revealed that when teams are automating core tests like UI/Functional, regression, end-to-end UI, and API tests, there’s a significantly higher likelihood that customer happiness is high as well. They indicated their customer happiness is pretty good or amazing, whereas teams not automating these types of tests rated their customer happiness as could be better or terrible.
So what does a culture of quality look like for leaders in DevOps? Naturally, QA professionals hold a leadership role in quality, but DevOps teams share responsibility across the entire team, especially as DevOps matures in the enterprise.
Furthermore for teams who have greater test coverage (really good or excellent), customer happiness is also impacted. 80% of teams who reported high test coverage also reported high customer happiness (pretty good or amazing).
With DevOps, a collaborative culture enabled by automation builds a pipeline and a process that teams have confidence in. When asked about their stress levels on release days, most DevOps teams say the process is manageable with little stress.
Almost one-third of DevOps teams even describe release days as smooth with no stress at all (must be nice, right?).
