We've teamed up with Detectify, an automated web application and domain security company powered by white hat hackers, to discuss how security and testing can be automated into your CI/CD pipeline. This article goes into how automated testing helps QA scale up productivity and practices with a DevOps environment, and to learn more about how automated security benefits DevOps, visit Detectify's blog.
Continuous integration and delivery continues to increase the flexibility and momentum of software product development. The results from a recent Atlassian survey include statistics that more than half of all software teams employ CI/CD in their environment. About one-third of survey respondents say their team has plans to move to CI/CD. The survey results also indicate that more than half of the respondent organizations can accommodate daily changes and deploy those changes to the end of the pipeline.
CI/CD improves developer productivity, and this translates into faster-evolving software products. Consequently, this adds more pressure onto the QA team. In a recent interview with mabl, a QA leader at a Fortune 250 company said that “Development cycles are getting shorter and new features are coming faster than ever. And we have five productive developers for every QA engineer. There's simply not enough time to test.” Part of the problem is that conventional testing tools haven’t kept pace with the improvements in software development and product delivery.
On many software teams, testers frequently find themselves in contention with the developers. Unable to keep up with continuous stream of changes and new features, QA becomes a bottleneck—and product velocity decreases. This unrelenting pressure often hampers any attempts by QA to improve processes or automate testing. There is no time to spend on initiatives that would improve throughput. In addition, the feeling of always lagging behind often contributes to low team morale.
Many agile teams don’t make an effort to write detailed functional specifications. The focus is kept at the higher conceptual level of user requirements. This often prevents the team from achieving consensus on a standard of quality. Because user requirements—typically in the form of user stories—are open to broad interpretation, testers often disagree with developers on both the validity of the test suite and also the severity of bugs.
Another issue is that many software product development teams frequently permit changes to requirements during the release cycle. When stories or requirements change, the test cases should change accordingly. However, more time is necessary for deeper engagement and coordination. And, this places more burden on team members that are already under stress. This forces QA to scramble and resort to ad hoc, unplanned exploratory testing.
Perhaps the biggest challenge of modern software testing is the amount of late-breaking changes to product requirements. Teams that have become more adept at managing dynamic requirements have implemented flexible CI/CD pipelines to increase their development velocity. Effective CI/CD requires accessibility and adaptability, so an effective test automation framework must integrate well into a CI/CD pipeline.
Modern development practices such as DevOps and Continuous Integration and Delivery (CI/CD) enable changes to be pushed more frequently than ever. However, antiquated test automation solutions are unsuitable for supporting DevOps and CI/CD.
They all require specialized scripting expertise, and given the lack of available talent, teams have very limited capacity for QA automation.
They provide limited debugging information in test results, which leaves teams with very little information to help them triage and address issues.
At scale, they require their own infrastructure, which takes time to provision, operate and scale.
They are too inflexible to use on a product under test that changes frequently resulting in constant maintenance and false positives.
These flaws only become more painful as the pace of development and change accelerates, compelling many teams to limit their use or discard them altogether, resulting in lower product quality.
There is a better way. At mabl, we’re empowering testers who’ve been left behind in the age of DevOps and CI/CD.
mabl simplifies automating functional UI tests by allowing testers to use their apps like their users would to create tests. Testers add test logic such as assertions as they go via an intuitive UI window. This record-and-play mechanic not only shortens the learning curve of test automation, but forces the tester to think like their user, and less like a computer navigating through an application.
mabl adds its own logic to fortify tests. An auto-healing feature identifies elements in the current release that have changed since the previous release, and updates the test script automatically. You can create tests that are accurate, effective, and largely self-maintaining.
mabl comes with a testing cloud that runs tests across all the major browsers, in parallel, on demand, with no limits. You can run entire test suites in the time it takes to run your longest test.
Once the tests complete, you get more than just a pass or fail notification. mabl gathers data on performance, visuals, the DOM and more, so that testers can easily share failures to developers with all the necessary logs for resolving the issue.
You can also closely integrate mabl with your CI/CD pipeline. Compatibility and integration with CI/CD pipelines is central to the design of mabl. Our mission is to improve testing to help all software product teams keep pace with the deployment speed of DevOps. With native integrations with the most popular CI/CD platforms like Bamboo and Jenkins, as well as Jira and Slack, teams can trigger mabl tests on deployment and send results anywhere they need.