“Your most unhappy customers are your greatest source of learning”
- Bill Gates.
Working with my teams, I’ve had some experience in gearing up for a big holiday sales season. I’d like to share the techniques and practices we used that may work for your team, whatever your busy season is.
In the eCommerce business, holiday sales will generate more revenue as many customers wait for this festive/holiday season to purchase what they need at discounted prices. Many eCommerce sites provide attractive discounts for the holiday season to grab the attention of flocks of customers. This will lead to far more traffic than off-season days.
It is vital to plan how you'll handle the traffic and to make sure your customers don't encounter any glitches or hiccups in their purchase experience. When parts of your website suck, it creates lots of opportunity for the customers to opt for your competitors' sites.
As an eCommerce tech team, these are the typical activities we plan before the holiday sale season starts:
- Form a team
- Collect the analytics metrics
- Prepare test scenarios
- Classify tests
- Set up the staging environment
- Set up automated regression test suites in CI/CD
- Prepare & execute performance testing
- Conduct capacity planning
I’ll take you through how my team worked through these activities to give you more info on the process.
Form a Team
We set up a separate cross-functional team that includes developers, testers, an eCommerce marketing specialist, a business analyst, and a project manager. Your team may look different, but make sure to set one up that can handle all aspects of the task.
Collect the analytics metrics
The next step is collecting the Google analytics metrics from the previous holiday seasons. This lets the team know the maximum number of users at the peak time of specific days, and understand the purchase behaviors of customers. With these insights, the team can decide where to focus our different testing activities.
Prepare test scenarios
Apart from the regular eCommerce site testing scenarios, we need to test the combination of promotional discounts. In general, there will be two levels of promotions applied. One is at the catalog price level and the second one is at the combo price at the shopping cart level. That is, when you buy a product “x” with product “y” then you will get a percentage discount off. Another popular promo is free shipping, and the list of promos to test goes on. We have to prepare these test scenarios and review them with the entire team to confirm we have the proper test coverage, along with the priority of the list of promotions planned for the holiday season.
Classify tests
Once we complete the preparation test scenarios, we classify tests into functional and performance test scenarios. Functional test scripts will be written and executed by the testing team.
Set up the staging environment
The next thing is setting up the staging environment. It is a replica of the production instance, but with the test data and a stable final release candidate. We populate the test data that simulates the production environment. Functional and performance testing scripts will be executed in this staging environment.
Set up automated regression test suites in CI/CD
We add updated automation regression scripts for the holiday sales to the existing CI/CD pipeline. Our goal is to ensure these run flawlessly ahead of time without any failures.
Prepare & execute performance testing
After writing our performance test scripts, we give them some dry runs in our QA environments, starting our simulations with a small number of users. Based on the metrics collected from previous holiday seasons, we decide on the number of users and the amount of load to be applied to the server during the tests. For example, a test run simulating user purchases may start with 50 users and be scaled up to a designated maximum of concurrent users. Reports of results are shared with the complete team.
Conduct capacity planning
Based on these performance testing reports, the capacity planning for production will be decided and an appropriate number of additional nodes will be added to the load balancer. With this, we will be better prepared and ready to handle the rush of holiday sales.
I hope these points will help you and your team make sure your eCommerce application is ready for the holiday sales!
Happy testing!
