The mabl blog: Testing in DevOps

Testing the Top 50 E-commerce Sites for Broken Links

Written by Eric Tatar | Jan 9, 2019 1:00:00 PM

Broken links can do a number on your site: they ruin your user experience, require redirects, and hurt your search engine ranking. The worst part is that most of the time, you don’t know they exist until you stumble onto one while clicking through old pages.

mabl's Free Link Crawler

As a safeguard against these broken links, all the websites you test with mabl are automatically checked with a free link crawler once every 24 hours. This crawler highlights any broken links as well as any Javascript errors it encounters so you can fix these issues before your users notice them: 

To see how some of the most powerful players in the web app space stack up (and after being inspired by this post by Pingdom where each website's uptime was tested), we set a mabl link crawler off on the top 50 e-commerce sites, according to Alexa. Alexa calculates the rank of each site using a combination of average daily visitors and pageviews over the past month.

To get more insight into the average performance of each site, we divided the elapsed link crawler runtime by the number of links checked to get an estimated average page load time for each site.

Before we get into the results, a quick disclaimer: For 9 of these sites, we saw that security against spammers, whether that be Captcha or other bot blockers that prevent the page from loading, stopped a sizable slice (18%) of the tests from gathering information. Another 2 sites were removed as outliers for a low page count from the tests and extremely long page load times, which could have been the result of more obfuscated bot blocking tools. If you use the kinds of blockers that could interfere with mabl on your site and whitelisting IPs is possible with your bot blocker, consider whitelisting mabl using its static IPs to allow our crawler to run, or run mabl in an environment where you may not have a bot blocker running, such as your staging environment.

 

Highlights of the results

The most impressive results came from Bed Bath and Beyond, where none of the 138,532 checked links were broken:

  Alexa's
  Ranking

 Site

Total
Links

Broken Links
41 Bedbathandbeyond.com 138,532 0
6 Etsy.com 52,042 1
7 Bestbuy.com 66,264 2
3 Ebay.com 16,010 3
38 Zappos.com 18,802 4
21 Humblebundle.com 37,242 10
50 Jcrew.com 5,792 10
45 Ulta.com 84,886 11
35 Dickssportinggoods.com 70,849 17


Nike had the most broken links, with 7,942, but this only constituted less than 8% of their 101,500 links checked:

Alexa's
Ranking

Site

Total Links

Broken Links

15 Nike.com 101,500 7,942
4 Amazon.co.uk 29,882 4599
1 Amazon.com 42,860 3865
37 Iherb.com 142,136 3,378
22 BHPhotovideo.com 22,315 2,127
44 Barnesandnoble.com 76,537 1,978
10 Ikea.com 8,935 1533
32 Oup.com 60,947 1501
12 Macys.com 78,891 1,298
23 Groupon.com 7,363 1,255

Most of the sites had a comparatively small percentage of broken links, with 84% of the sites having less than 5% of their links broken.

Ikea and Groupon took the top spots for the highest percentage of broken links, each hovering around 17%, with Groupon slightly nudging Ikea out of first place:

Alexa's
Ranking

Site

Broken
Links

% Broken
23 Groupon.com 1,255 17%
10 Ikea.com 1,533 17%
4 Amazon.co.uk 4,599 15%
22 BHPhotovideo.com 2,127 10%
1 Amazon.com 3,865 9%
15 Nike.com 7,942 8%
17 Costco.com 961 4%
11 Homedepot.com 729 3%
2 Netflix.com 598 3%
44 Barnesandnoble.com 1,978 3%

These sites also possessed two of the slowest average loading times per page; while the majority of the sites (89%) had an average page load time under 1 second, these two sites had page load times between 2 and 3 seconds:

Alexa's Ranking

Site

Total Links

Crawler Runtime (seconds)

Time per link (seconds)

50 Jcrew.com 5,792 21,602.00 3.729627072
46 Sky.com 6,704 21,587.00 3.2200179
23 Groupon.com 7,363 21,596.00 2.933043596
10 Ikea.com 8,935 20,453.00 2.289087857
19 Hm.com 13,079 11,642.00 0.8901292148
13 Cambridge.org 24,977 21,603.00 0.8649157225
34 Rebubble.com 18,736 16,135.00 0.861176345
3 Ebay.com 16,010 10,790.00 0.6739537789
22 BHPhotovideo.com 22,315 12,368.00 0.5542460229
11 Homedepot.com 22,180 11,914.00 0.5371505861

Interestingly, if we were to re-rank the top e-commerce sites based on performance and working links, our ranking compared to Alexa's would look like this:

Alexa's
Ranking

mabl's 
Ranking

Diffs

Site

41 1 +40 Bedbathandbeyond.com
6 2 +4 Etsy.com
45 3 +42 Ulta.com
35 4 +31 Dickssportinggoods.com
48 5 +43 Overstock.com
38 6 +32 Zappos.com
7 7 0 Bestbuy.com
21 8 +13 Humblebundle.com
25 9 +16 Lowes.com
3 10 -7 Ebay.com
40 11 +29 Rei.com
49 12 +37 Forever21.com
47 13 +34 Cargurus.com
29 14 +15 Shutterfly.com
20 15 +5 Nordstrom.com
34 16 +18 Rebubble.com
39 17 +22 Walgreens.com
30 18 +12 Sephora.com
50 19 +31 Jcrew.com
42 20 +22 Bodybuilding.com
16 21 -5 Wiley.com
19 22 -3 Hm.com
9 23 -14 Target.com
12 24 -12 Macys.com
8 25 -17 Store.steampowered.com
37 26 +11 Iherb.com
13 27 -14 Cambridge.org
46 28 +18 Sky.com
32 29 +3 Oup.com
15 30 -15 Nike.com
2 31 -29 Netflix.com
44 32 +12 Barnesandnoble.com
17 33 -16 Costco.com
11 34 -23 Homedepot.com
1 35 -34 Amazon.com
22 36 -14 BHPhotovideo.com
4 37 -33 Amazon.co.uk
23 38 -15 Groupon.com
10 39 -29 Ikea.com


You can see the full table showing all the sites' percentages of broken links here.
You can also see the full table of each site's average loading time here.

While most of these businesses have storefront locations, the experience of their online shops is paramount to their success as a company, as a growing portion of their revenues are coming from online sales. For smaller companies, a website may be the only place their users and customers can interact with them. This makes the identification of broken links and the assurance that a site can be used smoothly by all users a pressing matter for online shops.

How to test your website

Want to find out how your site measures up? You can create a free mabl trial account at app.mabl.com. The link crawler is run automatically when you add your test applications to mabl. 

Feel free to also follow these steps from the mabl docs to add more link crawlers, add restrictions to the crawlers, and more!