This week, mabl was a sponsor and speaker at Automation Guild 2021, the leading conference for learning about test automation strategies. As always, the conference was filled with a dizzying array of best practices for testing automation, tips and tricks for testing professionals, and wisdom from industry experts. 

The conference’s unofficial-official tagline reads “end flaky tests in 2021,” and at mabl, we certainly agree. Brittle, flaky tests continue to be a major obstacle for quality professionals in fast-paced companies, especially those operating in DevOps or CI/CD organizations. As a DevOps team ourselves, we understand the headaches that come when test automation frameworks fail, absorbing valuable time with high maintenance requirements. (That’s why we designed mabl with auto-heal capabilities, meaning mabl can automatically identify elements impacted by product changes and update the necessary tests without human intervention, drastically reducing the amount of maintenance required.)

To show what test automation looks like without flaky tests, Product Manager Anton Hristov and Software Engineer Lauren Clayberg presented how mabl tests with mabl at this year’s event. 

Testing Strategies at mabl 

Like any mature testing organization, the mabl team uses a range of tests to ensure full testing coverage: 

  • Unit tests: Unit tests are heavily used throughout all of our code bases with 66% coverage for the web app. Target 95% coverage for new projects.

    • (react-testing-library, Jest, JUnit, Mockito, chai, mocha, sinon)

Sample unit test report

  • End-to-end testing: We use mabl to test the end-to-end user experience of the web app with 100% page coverage.


Sample end-to-end report

 

  • Integration tests: Integration tests are also heavily used throughout our code bases and we use mabl for testing the UI.
    • (CodeceptJS for Trainer, Java for API, mabl for UI/API)
  • Exploratory testing: We do team-based exploratory testing (known internally as Elmer Fudds, except we’re hunting bugs instead of rabbits) as an extra precaution for the web UI, Chrome extension, and the native desktop app.

 

If you’ve visited the mabl blog before, you’ve no doubt seen the infographic below. It illustrates what a full DevTestOps pipeline looks like with testing integrated into every aspect of the SDLC. At mabl, this is also a model for our testing practices throughout the platform. 

Automated Testing Throughout Development

The mabl team has as many as 100 UI commits per week, a pace that would be near-impossible to sustain with brittle or flaky tests. The process begins during the coding stage, where we run unit tests and integration tests. For UI tests, a local runner is used to test the specific branch. 

Next, in the pull request phase, a preview environment is established and code is run through deployment tests and cross-browser tests. This ensures that the new code meets basic functionality requirements and reduces the amount of testing needed in the development stage. Our team has adopted a shift-left approach which enables our quality team to spend more time performing exploratory testing and engineers develop a better awareness of quality issues. 

In the development phase, all regression testing as well as exploratory testing is executed, which ensures that even the most ‘creative’ end users would be hard-pressed to break the app or website. In mabl’s DevOps workflow, the development phase also includes checking test coverage. 

Once code is in production, our team uses mabl to monitor performance and test coverage, particularly with our Segment integration that ensures testing is aligned with user needs. 

[To read about the DevTestOps workflow in greater detail, check out our Testing for DevOps Pipelines series!] 

Overall, the Testing in DevOps framework ensures that the mabl team is able to continuously test and deploy new code, while also continuously improving the mabl experience. 

Turning test automation into product quality

In addition to its advanced test automation features, mabl also has extensive monitoring capabilities. The team uses these to continuously improve our quality strategy with data. 

A major component of the mabl quality strategy starts with integrations, which allow testing to be fully integrated into the DevOps workflow, and ensures that issues are surfaced and managed efficiently. For a small team, this reduces the effort needed to keep everyone updated and frees up more time to focus on high-value development work. As we’ve grown, integrations with Slack, Jira, Segment, and GitHub continue to help us simplify quality management. 

The key to developing and maintaining an effective quality strategy is to be as data-driven and proactive as possible throughout the DevOps pipeline. At mabl, we practice what we preach to foster a culture of quality that not only provides the best intelligent test automation solution possible, but also addresses the challenges facing quality professionals in a DevOps world. 

Thanks to all who joined Anton and Lauren at Automation Guild 2021 and we hope to see you next year! 

To see how mabl can help your team keep up with fast-paced development, sign up for a free trial today.