Just a few years ago, our team at mabl set out to solve a growing problem for today’s agile development teams: web application testing. We focused on QA automation in particular because it is the most acute pain point for so many teams hoping to realize their full potential with DevOps. While there are dozens of solutions for testing application quality, most were built for a different era of installed, on-premise software that changed infrequently. These solutions are unfit for today’s complex, fast-paced development environments; they make it too hard for teams to collaborate on quality in a DevOps world. We had a vision to create a test automation platform that would fit seamlessly in this new way of working.
Our founding team saw that in order to compete and win against the incumbents, we would need to simplify the test automation problem for customers through the use of machine intelligence. We also knew that modern teams wanted to avoid the burden of installing, configuring, managing, and scaling a testing solution, so mabl had to be cloud-native SaaS.
After evaluating several cloud providers, we chose Google Cloud as our infrastructure provider largely on the strength of its serverless compute, data analytics, and machine learning offerings. Three years later, we could not be happier; we have assembled enterprise-grade services from GCP into a scalable core architecture with less time and money than building it ourselves, and this approach has enabled us to invest most of our human and financial capital into the innovation and user experience that are unique to our product.
Like many other businesses that have come before us, we’re currently in the middle of a significant market shift. COVID-19 has impacted businesses and economies around the globe, including ours. Building mabl as a cloud-native SaaS offering has given us the resilience we need, not only to manage through the fluctuations in utilization, but to be able to continue to meet the needs of software teams whether they have transitioned to all-remote, remote-friendly collaboration, or work together in the same location.
Here are five of the most significant ways Google has helped our team deliver and scale an enterprise-grade test automation solution in the cloud.
- Configuration management
From day one our team took advantage of several of GCP’s reliable and fully managed services. For example, we run tests on GKE, APIs on App Engine, process data in DataFlow, and more, without the added stress of server configs, patches, performance, and scalability. Using these Google services has allowed our team to keep operational overhead to a minimum, leaving more dev cycles for feature work.
- System operations
We use Google Cloud Monitoring for rich real-time metrics that help us understand customer usage and test performance. When we encountered our very first incident we had immediate access to the tools (Cloud Monitoring and Cloud Logging) we needed to quickly diagnose and resolve the issue. We have since instrumented GKE services with Cloud Monitoring custom metrics to build powerful dashboards that provide a deeper understanding of overall platform performance in real time; and include aggregated data across hundreds of thousands of time series to determine the overall health of our platform’s testing subsystems.
- Low latency network
For every networking need that our team has encountered there has been a world class solution offered by GCP - from Cloud DNS for domain hosting, to Cloud VPN for hybrid connectivity. Many software startups focus first on building their MVP, then focus on making it work at scale. At mabl, since the MVP was natively built on GCP, it simply scaled along with our business. For example, the core testing engine is built on Google Kubernetes Engine (GKE) and it has successfully scaled from the initial few testing cores to the thousands of cores it uses today, without modification.
Our customers are generating terabytes of test output. The mabl test automation platform writes this data to Google Cloud Storage (GCS) and the test artifact processing systems scale without modification to handle billions of test output files. Additionally, serverless GCP services like Google Cloud Functions and Cloud Pub/Sub, scale automatically to help analyze those outputs. The platform can process millions of work units at an economical rate, without any developer or operations intervention, simply because the architecture was built on highly scalable GCP services from the start.
- Scalable data processing
A differentiator of the mabl intelligent test automation platform is the rich diagnostic data available for every test run. Users can dig into every step of a failed test to identify and fix the root cause, quickly and easily. In order to provide this level of detail, we built the platform using Dataflow as an efficient way to continuously process a variety of data generated by tests, provide rapid access to distilled results, and enable retrospective analysis to drive future product decisions; and it incorporates auto-scaling to efficiently handle bursty workloads when many large test suites are being executed simultaneously.
We have also integrated BigQuery into the platform to capture and analyze data from both test executions and user activity in the frontend. It serves as a data warehouse where we can rapidly iterate on analyses and have a comprehensive picture of the platform from terabytes of raw data. Internally we use the information to help guide the business and make sound data-driven decisions ranging from how to improve core test execution capabilities, to monitoring new feature adoption. Externally, our customers can integrate their mabl workspace with BigQuery for more detailed analysis of their individual test runs and test coverage.
- Enterprise-grade security
The final area, arguably the most important, is security. It has been our practice to ensure the protection of all customer data at every phase of our evolution. While security can introduce complexity and overhead to software development, we have benefitted from being a modern, cloud-native company, building on the bedrock of Google and GCP’s two decades of cloud best practices.
Google Cloud Storage and Cloud IAM are the core components that help make keeping millions of customer files organized and secure, much more manageable. For example, all customer test inputs and outputs are stored in per customer Cloud Storage buckets. Using per customer Cloud KMS encryption keys, each customer bucket is independently encrypted for added security-at-rest.
Architecting with separate buckets has allowed simplified access control, auditing, and data cleanup. For example, if a GDPR request requires expunging customer data, removal of a single KMS key will immediately and irrecoverably crypto-shred the bucket and its millions of files. For comparison, in legacy cloud operations, complex database queries followed by millions of delete operations would be needed to identify, isolate, and remove all relevant files while that can now be done in a single API call on GCP.
Additionally, per customer buckets simplify access control decisions for both user file access and workload file access by eliminating the question of who owns what files. Using Cloud IAM Conditions and Workload Identity, mabl services can run under very tightly constrained Service Accounts, allowing adherence to the Principle of Least Privilege to access the minimum relevant customer Cloud Storage data needed for that service and customer.
We have also implemented GCP’s Uniform Bucket Level Access feature to ensure sensitive Cloud Storage buckets cannot have mixed access levels, so that all files are private. The platform is configured to rapidly detect if a bucket has been made public, and Cloud Security Command Center is used to notify operations staff immediately if a public bucket has been created, or the Uniform Bucket Level Access setting is absent. Instead of rolling our own security checks and notifications, we use GCP’s built in security services to reduce points of failure and reduce the engineering overhead of securing the mabl intelligent test automation platform.
The Google Cloud Platform provides the solutions that enabled our team to assemble a world-class infrastructure. We have been able to build a scalable, enterprise-grade test automation platform from the ground up by using these key capabilities, all while developing the next generation of intelligent test automation. To see mabl in action, visit https://www.mabl.com/trial-registration
