Get a Free Trial
Creating, executing, and maintaining reliable tests has never been easier.
Under a DevOps regime, deployments are intended to happen thick and fast. Top-performing DevOps organizations deploy up to four times a day, or 1460 times per year. Each deployment represents a functional change to software in production—in other words, this the last time you can test your software before your changes affect the end user.
The most mature testing organizations will have begun QA activities well before this stage. Starting with individual development activities, they would have started testing code during every save, every commit, and the PR approval process. Employing advanced automation will let them maintain the pace of deployment even while performing rigorous testing throughout the process.
Meanwhile, less mature organizations might only start testing during the pull request phase, which means that their testing activities will be most highly concentrated during the deployment phase. Without automation, testing will act as a brake on deployments, preventing developers from reaching the cadence of the most mature organizations (or their competitors).
Click the Diagram to ExpandAccelerating the pace of testing
Organizations that test primarily during the pull request and deployment stages can still optimize their testing processes to be faster and smarter.
One great example of this occurs during PR approval. Before official full-scale deployment, our advanced users will often deploy changes to what’s known as an ephemeral environment. Right before pull request approval, an automated tool spins up a QA environment with an instance of the application that’s accessible through a browser. This lets developers perform full regression testing—both automated and manual—before any changes are merged with the main branch. Meanwhile, the environment evaporates the moment you merge —saving your budget from getting eaten up by cloud costs.
Using ephemeral environments gives developers the opportunity to perform tests that weren’t available during the code stage or the early pull request stage. Cross-browser testing, for instance, will only be viable in this kind of ephemeral environment. That being said, many developers prefer to wait until the deployment stage to perform this kind of test.
Automated testing in pre-production environments
As mentioned, the deployment phase is where many developers choose to concentrate their testing. Without automation, testing can delay time-to-market— making automation a mission-critical capability. Keeping this in mind, we’ve designed mabl with a heavy focus on the deployment stage, with the aim of letting developers perform intensive and meaningful testing in a relatively short time.
At this point, code has been merged into the main branch and it is being deployed into one or more pre-production environments. Using mabl, we’ve created a staging environment. After the pull request is merged, it’s tagged and then an automated process places the new main branch into the staging area. More mature developers will treat staging as just one among several environments—they will have environments for development, QA, and staging, each with its own array of testing functions.
Since these environments are real—in that they’re hosted on the internet and can be accessed via URL—they are used as a platform for many different kinds of manual and automated testing. These environments are also persistent. To maximize their value, developers should be continuously running tests.
An ideal workflow involves setting up a collection of automated tests that run every time developers merge changes into the main branch. These are built to run as soon as the changes arrive in the staging or QA environment, and they run not just feature-level tests, but the entire regression suite. This can add up to hundreds of tests, but since they’re set up to run automatically and powered by the speed of the cloud, they still run fast enough to match the cadence of DevOps.
What wouldn’t you test? The only trade-off with automation is with test environments hosted in the cloud, running tests costs money. In order to save time, you could avoid running tests on every single browser or you could avoid running the longest-running tests, or a myriad of other options. What we typically see is that even though developers avoid running some tests every time a change gets dropped in, they’ll still run their entire test suite overnight in accordance with best practices.
Align towards testing best practices with mabl
Here at mabl, we know that not every organization is at the same maturity level of automated testing. That’s why we’re devoted to making sure all organizations can run effective tests at all stages. With advanced low-code automation and a user-friendly testing interface, we can make sure that every DevOps organization tests to the maximum of their ability, resulting in better-performing applications and happier customers.
This blog is the third entry in our series on testing for DevOps pipelines. To learn more, check out our posts on understanding the code stage, pull request testing, and testing in production.
To see for yourself how mabl can help improve testing in pre-production and through the entire development lifecycle, sign up for your free trial today!
