How Stack Overflow Used Automation to Test Complex Applications

If you’re in software development, you know Stack Overflow. The beloved platform serves over 100 million developers every month, helping people build their skills, share knowledge, and grow their careers. But maintaining a complex and mature product like Stack isn’t easy, especially as the team introduces new offerings like asynchronous knowledge management and collaboration tool Stack Overflow for Teams. As a small team with big impact, Stack Overflow needed a software testing and quality engineering strategy that helped them both maintain their 13 year old codebase and move quickly to grow new offering.Their search for a new software testing solution led them to mabl. 

VP of Product Engineering Dale Cook and Software Engineer Meredith Hatzenbuehler shared how test automation is enabling the Stack Overflow team to build a better software development organization and improve product quality at mabl Experience. 

Maximizing the Value of Software Testing with Test Automation

Stack Overflow’s mature codebase and growing product offerings meant that Dale and Meredith had specific needs for a new test automation solution. The team needed a test automation that could match the pace of development, yet accommodate the nuances of their complex products. Meredith began experimenting with different solutions to better understand how each would fit into Stack Overflow’s engineering processes. After exploring a range of options, she and Dale made their final decision.

Dale explained their criteria: ”We found that mabl was a clear winner. There were other software testing tools that allow for the same easy test creation that mabl does, but they weren't performing. They were slow in the way that they work, so although we could build tests really fast, the tests themselves were not particularly fast. But the thing that really sold us on mabl was its ability to integrate with our engineering workflow. Not just with building tests, and getting mabl integrated into our CI/CD pipelines and that kind of thing. But integration with quality engineering workflows, so that engineers can be running mabl tests, not just the unit test, not just their integration tests, but actual mabl tests as part of their workflow. That was a really powerful thing.” 

The full benefits of mabl; easier test creation, faster test execution, and more integrated workflows with Stack’s CI/CD pipeline, have made it possible for Meredith and Dale’s team to make concrete progress towards their goals. 

Less Time Software Testing, More Time Building

You may not realize it as a Stack Overflow or Stack Overflow for Teams user, but the engineering team is relatively small compared to the complexity and impact of their products. As many software pros know, an efficient test automation strategy can be the make-or-break factor in introducing new features frequently, or being stuck in endless production delays. When rethinking their software testing partner, Dale and Meredith were acutely aware of maximizing engineering resources. 

As Stack Overflow’s VP of Product Engineering, Dale was particularly focused on using test automation to help engineers focus on building, not fixing: “We want engineers to be building new things and doing new things that they’re interested in. How do we build a really robust testing platform for a complex product without having to take a lot of engineering resources?”  

Implementing mabl’s low-code test automation platform allowed Stack Overflow to create a virtuous cycle of testing - bug discovery - bug resolution - higher quality deployments. With a better software testing process backed by test automation, they’ve been able to create a highly efficient testing workflow that makes collaboration much easier. 

Communication, Collaboration, and Replication

Before most of us were working in sweatpants and worrying about which Zoom backgrounds to use, Stack Overflow was a distributed team with colleagues around the world. Creating sustainable workflows that can adapt across different time zones and reduce miscommunications was a must, especially when it came to product quality. Mabl’s robust set of integrations and reporting features proved to be an unexpected resource for managing the handoff of tests and defects. 

Meredith noted that her team relies heavily on the contextual information that can be added to automated tests in mabl: 

“I've been surprised by how much value we've gotten out of things like screenshots, test steps, network information, the DOM, et cetera. That’s helped us collaborate with other teams. For example, if an SRE needs to step in, I can simply give them mabl credentials so they can log in, and look at the test. And they’ll have exactly what they need. Whereas before, there would probably be back and forth discussions since we wouldn’t always have such a detailed record with other types of automated testing. That loop of solving problems has definitely become much more efficient.”

More efficient test automation leads to faster bug detection and, ultimately, a better software development organization. The next challenge is scaling Stack Overflow’s new quality engineering strategy across the entire product. 

For Meredith, scaling with quality means diving into the mabl features designed to make your test automation strategy easy to grow and manage. Reusable flows helped her team shift focus away from a product-level focus to a component-level focus, which was a much more efficient approach as many components have been reused across multiple Stack Overflow projects in its 13 year history. Though her team is focusing on automating tests for the Stack Overflow for Teams product, the library of reusable flows will help other Stack colleagues quickly get automated testing underway for their own projects.  And when they do, they’ll have an organized, easy-to-navigate mabl workspace established by Meredith to make quality engineering adoption seamless. 

Building a Better Engineering Organization

So what’s next for mabl at Stack Overflow? Dale shared his thoughts: 

“We continue to think about how we can make a better product for us internally, so that it's easier for engineers to build things, adopt more automation, and maintain a better codebase.  That kind of stuff allows us to move faster in our product releases. So over the next year, we're going to spend a lot of resources doing that, which we're very excited about. We think it's going to have an incredibly important impact on the future growth of Stack Overflow.”

Register for mabl Experience 2023 to see industry leaders like Dale and Meredith share their stories of software testing and quality engineering.