The Five W’s (and an H) of Continuous Testing

Breaking the software testing silo has been a hot topic in DevOps for years. Yet many teams are still struggling to successfully shift-left, shift-right, or otherwise meaningfully transform when and how software testing is performed during the software development process. Enter continuous testing, where code is monitored for errors from the very start of the development cycle. 

For those still testing primarily at the end stages of software development, continuous testing might seem like being asked to run a marathon before finishing a couch-to-5K program. Luckily, mabl Software Engineer Joe Lust broke down the 5 W’s of continuous testing at mabl Experience. With these pillars in mind, quality engineering teams can understand and plan their journey to continuous testing. 

Why is Continuous Testing Important to Quality Engineering?

Software engineers need high velocity, high throughput pipelines for delivering code to production and for deploying changes. The accelerating rate of innovation throughout the software industry means any software company looking to remain competitive needs their developers to make those changes quickly…without introducing errors to the product.   

However, the current software testing strategies are often a challenge for organizations trying to accelerate development, as demonstrated by the fact that  43% of software developers say testing is their biggest pain point. The old model of testing in a silo is simply not effective in the world of DevOps and agile. Which is why software development organizations are turning to continuous testing to accelerate and expand their quality engineering efforts. 

Continuous testing throughout the software development pipeline ensures that changes can be introduced quickly without sacrificing quality. But how (we warned you there was H) can quality assurance teams start their journey to continuous testing?

When and How Should You Use Continuous Testing? 

Let’s start at the beginning: the code stage of development. Engineers may quickly introduce new features thinking they’re adding value to the product, but they may also be unintentionally introducing defects. Testing at this point in software development can help reduce errors without adding friction to engineer workflows. Defects caught this early in the development cycle are also significantly easier to fix, enabling faster development without adding risk. 

“We need a better way to help engineers isolate these bugs early, at the first stage in our pipeline,” said Joe. 

Though early testing is a good starting point for an effective testing strategy, limiting testing scope to this stage is not enough. Instead, Joe suggests that engineers never stop testing. 

“The DevOps symbol looks like an infinity symbol, and that’s no coincidence. Quality never takes a break. So we need to test whatever defects could occur. Things could break at any time. So, we don’t just want to test when we make a code change. We want to test continuously,” he said.

So while introducing testing at the code stage is an important milestone in the journey to continuous testing, it’s hardly the only opportunity to expand a software testing strategy. 

What does Continuous Testing Look Like, and Where Are We Going to Use It?

When automated testing is integrated throughout the development process, it’s critical to make software testing as seamless as possible. 

Developers can deploy headless local test runs in the background of their work, so they’re not interrupting their flow states. With tools like the mabl desktop app, an engineer can easily run those tests in about 10 seconds.

“What I just did here is I checked another box that said run headless. We’re not going to pop up anything in the way. It took five seconds to run, and I have high confidence in that code change.” Joe said. 

Developers can also perform smoke, API, visual, regression, and cross browser tests with limited disruption to their existing workflows, making it easier to adopt continuous testing. 

“These tests give us feedback directly into our review context to understand how good or bad a change is. That sets us up for long-term success, because as we all know that the earlier we catch a bug, the less time it takes to fix it,” added Joe. “We make sure that we have high confidence in this application change and also a broad, clear picture of the quality of the change.”

Deployment collects all of the quality information about the change engineers are considering moving to production, offering them a broad picture of its quality. So, they can see all of the rich insights and labels collected before deciding if they want to promote the change.

The final stage in a change pipeline is transferring the change to production for user impact. However, this stage shouldn’t be the conclusion of continuous testing, which should move into production as well. Instead, engineers can use those tests they created earlier as monitoring tools that continuously validate production. 

“You just use your scheduled plan runs. You can tell it to run, for example, every day at 2am, or every hour. You simply attach them to whatever your monitoring tool is,” suggested Joe. “Once you build all these rich end-to-end tests, you can simply integrate with your pipelines with these schedules and different events, and you get that continuous testing at all stages of the software development pipeline.”

Who Can Be Part of Continuous Testing?

Everybody is a participant in quality engineering with mabl. With the rich information made available, every member of the team has screenshots, console logs, and other information at their fingertips. 

“If I’m an engineer, it’s as if I’m right there at that computer when the user experience is the issue,” said Joe. 

Joe has seen mabl's non-coding users, including business analysts, product managers, and other management groups, log in, see the results of tests and coverage reports, and make simple changes. 

“Now, everyone has been able to contribute to quality. That frees our quality engineering group to focus on higher-level, objective quality tasks. We really see this as a way to democratize testing, so that everybody in your organization can participate,” he remarked. 

The Benefits of Continuous Testing

Continuous testing establishes the foundation for a culture of quality, where development cycles are accelerated without introducing bugs to the user and experience, and everyone is an active participant in your quality engineering strategy. This makes it easier to continuously improve overall product quality with quality engineering. 

Mabl is designed to help software development teams unlock continuous testing, no matter their current workflow or automated testing strategy. Integrations with Jira, Slack, Microsoft Teams, and other popular development tools make it easy to adapt continuous testing to your team’s preferred development processes for sustainable quality engineering practices. 

“If you ever use tools other than mabl, you know you have to install tons of components, get them all configured just right, and get everything running just right before you can test. Test automation with mabl is an amazing resource because if you can integrate testing every way possible.” said Joe. 

Get started with low-code test automation that unlocks the power of continuous testing with mabl’s 14-day free trial.