Building Intelligent Pipelines with mabl and GitHub Actions 2.0

At mabl, we have long envisioned a world where it was easy to configure and operate intelligent CI/CD pipelines. In that world, new code changes are automatically tested and assessed for their quality and risk. If the risk is too high or the quality is too low, the changes are automatically held back until the team takes appropriate action--by fixing bugs, adding test coverage, reviewing changes more thoroughly, or otherwise. If the risk is low and the quality is high, the changes are moved along in the pipeline quickly and efficiently. 

In order to achieve that vision, we recognized that testing tools would need to become more intelligent. For the past two years, mabl has focused on building the most intelligent end-to-end testing service on the planet. Today, people can use mabl to create tests in minutes, run them on all major browsers, and receive insights about their application quality, including functional correctness, visual presentation, performance, and more. 

To achieve the vision of intelligent CI/CD, we also recognize that pipelines would need to become more integrated. Today, GitHub launched Actions 2.0, which we believe will serve as a leading platform for smarter, more integrated CI/CD workflows. We jumped on the opportunity to integrate with Actions 2.0 and GitHub more broadly. In this post, we'll show you how those integrations work.

Overview of the
GitHub / mabl workflow

At the highest level, the workflow starts with you making code changes within a given GitHub repo, building them, and deploying them to an environment using Actions 2.0.  At that point, a second Action triggers end-to-end tests within mabl. If mabl discovers problems during the test runs, it creates corresponding GitHub issues within the relevant repo.

The challenge

Our challenge was to automate the most common activities related to testing in a CI/CD pipeline; kicking off automated tests and tracking issues. We wanted to deliver those capabilities in a manner that was easy to configure for new users but was also flexible enough to support the needs of power users.

Testing new deployments

In order to automatically kick off tests of new deployments, we created a JavaScript Action using Actions 2.0 that would call mabl’s deployment events API when new deployments occur. In addition to specifying the application and environment for the tests, the Action can also specify which browsers should be used for the tests, whether the current build should be considered the “baseline” for visual analysis, and more.

When mabl receives a deployment event, it automatically triggers the relevant tests based on the parameters that are specified in the Action. The Action then polls mabl to get the status of each of the triggered tests until they are all complete. At that point, the Action completes.

By default, if any of the triggered tests fail, the Action returns a failure and stops the pipeline.  Alternatively, you can configure the action to continue on failure appending the following:

Part of the power of this model is its flexibility. The mabl action uses the outputs feature of Actions 2.0 to output a number of statistics about your test results. You can use these statistics to control which subsequent actions are executed. For example, rather than failing the workflow if any test fails, you can customize the action to only fail if more than five tests fail by appending the following:

You can even configure your pipeline to automatically promote the build to the next environment if the tests all pass, so that you don’t need to manually tag or trigger a push to production. In this example, we call another action to deploy the change to production:

When the Action receives information on test results, it also provides you with deep links into mabl to view the detailed logs, screen shots, DOM information, and more.

Actions log with links back to mabl output

We are so excited about this integration because you can be up and running with the core functionality really quickly, but it also gives you the flexibility to tailor the workflow to meet the needs of your team over time. 

Creating issues automatically

In order to streamline issue management, we created a simple mabl GitHub app that you can configure to read your mabl event stream and create issues automatically based on the results of tests. This saves you the time of collecting the relevant information and manually creating each issue by hand.

You can also configure the integration to create issues when mabl detects Visual Changes, JavaScript Errors, Timing changes, Broken Links, and more.  

GitHub integration settings

The issues include screenshots from the test and links back to the detailed test results in order to help you triage and resolve issues as quickly as possible:

A GitHub issue with the latest screenshot and a link to the full mabl test output 

Bringing it all together

In order to make the experience truly seamless, we also wanted to make it easy for you to navigate across the GitHub and mabl interfaces. Fortunately, GitHub makes it easy to maintain the appropriate context across the integrations; the mabl action passes PR, committer, and branch information along to mabl automatically. As of today, you can filter your mabl test runs based on Commit information from GitHub.

Filter mabl test results based on commit information from GitHub

We're working hard to add deeper links between mabl test results, pull requests, branches, and more.

Next steps

This is the first in a series of integrations and enhancements that we have planned to help you realize the potential of intelligent CI/CD pipelines, so please follow us on Twitter to stay up to date.  In the meantime, you can sign-up for the GitHub actions Beta and get your free trial of mabl to try it out for yourself!

Watch a quick overview of the new mabl GitHub Integrations: