Get a Free Trial
Creating, executing, and maintaining reliable tests has never been easier.
Modern software development requires a much shorter time from conception to delivery than it has in the past. In order to ensure that quality is maintained within much shorter sprints, teams have started to incorporate testing into earlier stages of the development process. This way of moving testing early into the development process is known as a “shift left” approach.
The problem with traditional testing approaches
Even though many have moved beyond the traditional waterfall approach to software development, software testing is oftentimes left to the end of the cycle.
It’s no fault of developers that testing is saved for last. In fact, many developers want to test more, but they’re incentivized not to - with tight deadlines and conflicting priorities that limit how much they can step back and focus on their testing strategy. There are many obstacles that make it a burdensome process to test throughout development. However, we know that by not doing so, we create a bottleneck toward the end of the development process which has the effect of slowing down releases or leaving them lightly tested.
Shift left encourages a culture of quality
Shift left isn’t only about involving QA early in the development process; it’s about democratizing software quality across the entire team. This “culture of quality” should extend from development, to QA, through to ops. Software testing is always going to be a heavy lift, so it’s simply unrealistic to treat it as just another task that can be lessened or brushed over when time constraints rear their ugly heads. Quality assurance needs to be viewed as an equally crucial part of building the software as development.
Testing is not just meant for identifying bugs, but is designed to build a clear strategy for improving quality throughout the entire process.
In a shift left approach, we allow testers to design the tests in the design phase of development before any piece of code is written, focusing on UI and UX principles. Dedicated testers should always be a part of each development team. However, in a shift left approach, testing is not just limited to testers; developers are expected to write tests as part of their development processes.
But we know that it’s not easy or realistically possible to have the right team structure and processes built up overnight. What can teams that don’t have this setup in place do to get the benefits of “shift left”?
We spoke to several developers about their testing habits. All of them stated that their team had to have unit tests before submitting a PR. What they had to say about that process was eye opening.
The traditional testing triangle
There are a wide range of different testing types, including unit testing, integration testing, regression testing, end to end testing and far more. In a “shift left” approach, it makes sense to move some, but not all types of testing leftward in the development process. We’ll discuss which tests can be shifted left below, and we’ll examine how these different testing types “stack up” in the form of a pyramid, where the widest area at the bottom depicts where you should focus the greater number of tests.
The traditional testing triangle suggests that the vast majority of tests should be unit tests. After individual units are tested, integration tests are performed to make sure each unit works with the others. Only after most development has been completed are end-to-end tests performed, and most of these end-to-end tests should be done manually. We won’t dive too deeply into the philosophy of why in this article, but you can read more about it here.
Complications with the traditional testing triangle
Unit tests
Unit tests verify individual components of code. Popular architectures are moving away from monolithic applications and moving towards highly decoupled, focused services, such as microservices. Many times these services will only have one function, meaning there isn’t much for a unit test to verify. One developer explained:
“Methods can be so simple that sometimes unit tests can be useless. Unit tests are faster to write, but they only test a tiny piece of the puzzle. They’re the ones you write when you have a time constraint: ‘It’s the bare minimum testing but at least we have it’ kind of thing.”
The payoff of unit tests comes early on, but they can become a burden in the long run. If code needs to be rearchitected after a code review, the unit tests will also need to be rewritten. As a result, many development teams have found a better practice in writing unit tests later in beta stages of development - shifting testing further to the right instead of the left. One of the developers we spoke to remarked:
“Unit tests aren’t hard to write, but they’re subject to change if we re-architect our code or code reviews come back with suggestions on how to change it, in which case we’d have to rewrite the unit tests anyway.”
Integration tests
Integration testing is a more effective type of testing for modern applications, where you’re testing SLAs and verifying that the communication between different services is working as expected. However, upleveling unit testing to integration testing becomes a pain to do due to the nature of microservices. This means writing integration tests for modern applications require extra work once complex networks of communication between microservices start happening. The same developer explains:
“At the integration testing level, tests are less subject to change, but they’re a pain to write. For example, in order to write integration tests for a microservice, my microservice might communicate to five other services, where I’d have to stub dummy data for those five other services. It’s a lot of overhead and hassle.”
End-to-End tests
An end-to-end test looks at the entire system together and verifies a whole user-story, rather than just an integration point. End-to-end testing is its own specialty; to automate effective and reliable end-to-end tests requires someone with both a strong overall understanding of business objectives and strong engineering skills. For this reason, the best approach for writing effective end-to-end tests is to integrate team members with testing and coding skills and subject matter expertise from the beginning of development. But that’s quite a steep ask - someone with that resume is incredibly hard to find.
While difficult to implement, the value generated by end-to-end tests is worth more than the time they take to create:
“We are thoroughly convinced in the value of end-to-end tests and integration tests, and we are working towards a world where we require those as part of our PR.
Integration tests do a pretty good job of validating that your code is meeting the SLAs of talking to other pieces of the app, for example, an HTTP server not responding.
But end-to-end tests show business logic errors or when things just look weird - what our users see. That’s difficult to catch with integration tests.”
The Shift Left testing triangle
After speaking to a group of developers about their testing strategy, we found that the most effective type of tests to shift leftward are end-to-end tests; they catch issues that inherently expose integration issues, and they pivot the testing approach away from low-level code to business-critical requirements.
In a perfect world, software teams would include end-to-end tests to cover every aspect of an application, encompassing all regressions that may occur and giving us a new testing triangle that looks like this:
But we can’t ignore that end-to-end tests are expensive and take a lot of resources because of the unique and demanding skillset they require. Another developer explains further:
“When working on a service, I know I may have 10-12 endpoints that my service will listen in on; that wouldn’t be so bad to write end-to-end tests for, especially when we’re just adding tests as we go. The difficulty is when we end up with multiplicative complexity and different states. It took us a year and a half to write end-to-end tests for a test suite for our product.”
It was fixing this exact pain point that we designed mabl for.
Using mabl to shift end-to-end testing left
After listening to each developer dive deep into their testing pains and goals, I asked them what would make end-to-end testing easier. Not surprisingly, many listed these points:
- Increasing the speed of writing tests so that I can continue to meet my tight deadlines
- Having a highly configurable tool so that I can write tests for the many complex scenarios and application states that I need to test
- Making it easy to use so that the team doesn’t have the bottleneck of relying on one expert on the team to create these tests
Mabl meets all of their needs. It reduces the complexity of test automation scripting by allowing developers to create end-to-end tests by simply using their apps like their users would. mabl is tech agnostic and has dozens of built-in advanced features for testing complex apps and scenarios. mabl is easy to use while being so feature-rich that many of our customers have reduced the time it takes them to write an entire test suite from over a year to a few weeks. You can check out our customer stories to see exactly how that’s possible.
If you can relate to these testing pains, try creating a free mabl account right now and see how easy it is to test your complex scenarios.
