This article is from a webinar series for mabl customers by Matthew Stein. You can find the video at the bottom of this article.
It’s worth putting some time into grouping your mabl Journeys together so you can efficiently run the right ones at the right time. Today we’re going to walk through several methods to group them in a way that matches your organization structure.
Let’s start by covering a few terms.
Journeys:
Journeys are the way we refer to a test trained in mabl with specific steps in a web application or website, since they’re meant to mimic the “user’s journey”. You’ll read a lot about journeys here.
Plans:
Think of a Plan as the parameters of the test suite. Journeys run within Plans, and the Plan specifies WHERE and WHEN journeys will run. Plans can be differentiated by environment (Prod, staging, dev, etc) or by URL.
Triggers:
Triggers schedule plan runs. mabl has three different types that you can use.
1: Interval triggers: You set the amount of time between runs. If you select 4 hours, and kick off the Plan at 2pm, then the next run will start at 6pm.
2: Schedule triggers: Pick the day of the week and the time of day you want the plan to run. You can select individual days (i.e. Monday, Wednesday and Friday) or you can choose to run it every day. Then you select the time of day, and the Plan will run at that time on each of those days.
3: Deployment triggers: If you integrate mabl with Jenkins, Bamboo, or your own CI/CD tools, you can run your plans on your deployments. Set the plan you want to run on deployment. When your developers trigger a deployment, the Plan you picked will run. Here’s more on triggering plans via the API.
Now that you understand how Plans run, you can choose what type of Journeys to include in them.
End-to-end tests. End to end tests help make sure that newly shipped features don’t break an overall customer or user flow. These focus on “happy-path” tests. You start with a complete user interaction, make assertions, then run some cleanup. Start by training all of the steps for that interaction. Then add your assertions to ensure that the data is being handled or created correctly. You can set these assertions to “continue on failure” if you want mabl to keep going through all of the steps. Otherwise, mabl will stop executing the Journey on a failure. After your assertions, you can run clean-up or tear-down steps. If your end-to-end journey involves creating a new customer, then creating an invoice and asserting that the data is correct, you probably want to remove those test customers and invoices from your database. Train some clean-up steps at the end of your Journey so you don’t clutter your database with extra objects. These make for a well structured end-to-end test.
Your developers are probably creating unit tests for the features they’re working on. That’s a common practice. Even if the unit test passes, you want to ensure that the new feature doesn’t break anything further upstream or downstream. A good end-to-end test covers the way that feature interacts with the rest of your application.
Smoke tests are short tests to verify critical actions. After you ship a new feature, you want to ensure that the core functions of your app are still working. For an e-commerce site, your smoke tests would include functions like: logging in, searching for an item, adding it to a cart and checking out. You should build these smoke tests to be focused and as simple as you can. Put all your smoke test journeys in the same plan, since mabl can run your Journeys in parallel, which means your smoke test suite can run as quickly as possible.
Smoke tests can also be used to block or pass deploys from shipping.
Regression tests. Truth be told, all the tests you create with mabl are considered regression tests. A full suite of regression tests can be focused on one particular aspect of your app. Train your journeys so that (with their forces combined) they test the primary use case, edge cases, and ensure improper use cases are caught. When training regression tests, you should consider using re-usable flows for efficiency.
Organize by Team: Organizing your plans by team helps you identify the people associated with the tests. These days, most development work happens in the context of a team. Unless you’re hacking away at a brand new startup, you’re probably working with other people. If developers are working together, it makes sense to run tests in the same groupings. This enables monitoring and asserting ownership of the Journeys and their results. If a Journey starts failing, you know who to talk to and get it working again. If you have a dedicated QA person working with the scrum team, they know exactly what tests they are working on and where to find them.
Use labels to identify Plans by team or individual. If you have a lot of plans, then labels make it easy to quickly find the one you’re looking for.
Organizing by User Role: This helps you test differentiated feature sets. For example, your application could have user roles like administrators, editors and viewers. Admin roles might be the only ones who can modify billing invoices or change prices. Viewers could search for customers, but not edit them or modify invoice amounts. Start by training Journeys which verify those user-specific features. Log in as an admin, navigate in your app, and assert that admin-only features or links are there. After that, build a set of tests for “viewer” roles and assert that they don’t have access to admin-only features. This tests that the functions assigned only to certain roles are unavailable to other roles. Training a Journey to assert that edit buttons aren’t available for your viewer roles gives you an on-going assurance that these roles are properly secured.
Why group Journeys together by user role? Why not just add all of the user role journeys together in the same plan? Because the plan stores a set of credentials to use for logging in. Save yourself time in the future by grouping them together, so when you need to update the passwords you only have to change it once. Each journey on the plan will use the new credentials.
Organizing by feature sets: Grouping your journeys is all about deciding what needs to be run at the same time. When your team is working on a particular feature, it’s helpful to have a suite of tests that covers all the scenarios for only that feature. You want to isolate running those tests from all the other general coverage tests.
Organizing by Navigation: If you’re testing a website instead of a web application, this might be better for you. Your website is organized by its navigational structure. You can mirror that same structure by grouping your Journeys the same way. Each top level navigation item can have its own Plan. Then, as you’re working on the site, you can easily find and run the appropriate suite of tests for your current project.
Organizing by JIRA Project: This makes sense for some organizations. Chances are, if you use JIRA, your development team already thinks in terms of JIRA projects. It’s only logical to group your journeys into plans that follow the same structure. Regression tests grouped under the same buckets make visibility very clear.
The main question to consider is which tests need to run at the same time. This is the single most important concept to think about as you organize your mabl workspace. Journeys that share information like credentials or a starting URL should go together so you can update them all as necessary. Integrate your CI/CD tools with mabl to automatically kick off the right plans on deployment events.
What’s your strategy for organizing journeys and plans? Share it with us and we might feature your ideas in the next webinar on this topic.
You can also read the slides on SlideShare here.