Broken links can do a number on your site: they ruin your user experience, require redirects, and hurt your search engine ranking. The worst part is that most of the time, you don’t know they exist until you stumble onto one while clicking through old pages.
mabl's Free Link Crawler
As a safeguard against these broken links, all the websites you test with mabl are automatically checked with a free link crawler once every 24 hours. This crawler highlights any broken links as well as any Javascript errors it encounters so you can fix these issues before your users notice them:
To see how some of the most powerful players in the web app space stack up (and after being inspired by this post by Pingdom where each website's uptime was tested), we set a mabl link crawler off on the top 50 e-commerce sites, according to Alexa. Alexa calculates the rank of each site using a combination of average daily visitors and pageviews over the past month.
To get more insight into the average performance of each site, we divided the elapsed link crawler runtime by the number of links checked to get an estimated average page load time for each site.
Before we get into the results, a quick disclaimer: For 9 of these sites, we saw that security against spammers, whether that be Captcha or other bot blockers that prevent the page from loading, stopped a sizable slice (18%) of the tests from gathering information. Another 2 sites were removed as outliers for a low page count from the tests and extremely long page load times, which could have been the result of more obfuscated bot blocking tools. If you use the kinds of blockers that could interfere with mabl on your site and whitelisting IPs is possible with your bot blocker, consider whitelisting mabl using its static IPs to allow our crawler to run, or run mabl in an environment where you may not have a bot blocker running, such as your staging environment.
Highlights of the results
The most impressive results came from Bed Bath and Beyond, where none of the 138,532 checked links were broken:
Alexa's |
Site |
Total |
Broken Links |
41 | Bedbathandbeyond.com | 138,532 | 0 |
6 | Etsy.com | 52,042 | 1 |
7 | Bestbuy.com | 66,264 | 2 |
3 | Ebay.com | 16,010 | 3 |
38 | Zappos.com | 18,802 | 4 |
21 | Humblebundle.com | 37,242 | 10 |
50 | Jcrew.com | 5,792 | 10 |
45 | Ulta.com | 84,886 | 11 |
35 | Dickssportinggoods.com | 70,849 | 17 |
Nike had the most broken links, with 7,942, but this only constituted less than 8% of their 101,500 links checked:
Alexa's |
Site |
Total Links |
Broken Links |
15 | Nike.com | 101,500 | 7,942 |
4 | Amazon.co.uk | 29,882 | 4599 |
1 | Amazon.com | 42,860 | 3865 |
37 | Iherb.com | 142,136 | 3,378 |
22 | BHPhotovideo.com | 22,315 | 2,127 |
44 | Barnesandnoble.com | 76,537 | 1,978 |
10 | Ikea.com | 8,935 | 1533 |
32 | Oup.com | 60,947 | 1501 |
12 | Macys.com | 78,891 | 1,298 |
23 | Groupon.com | 7,363 | 1,255 |
Most of the sites had a comparatively small percentage of broken links, with 84% of the sites having less than 5% of their links broken.
Ikea and Groupon took the top spots for the highest percentage of broken links, each hovering around 17%, with Groupon slightly nudging Ikea out of first place:
Alexa's |
Site |
Broken |
% Broken |
23 | Groupon.com | 1,255 | 17% |
10 | Ikea.com | 1,533 | 17% |
4 | Amazon.co.uk | 4,599 | 15% |
22 | BHPhotovideo.com | 2,127 | 10% |
1 | Amazon.com | 3,865 | 9% |
15 | Nike.com | 7,942 | 8% |
17 | Costco.com | 961 | 4% |
11 | Homedepot.com | 729 | 3% |
2 | Netflix.com | 598 | 3% |
44 | Barnesandnoble.com | 1,978 | 3% |
These sites also possessed two of the slowest average loading times per page; while the majority of the sites (89%) had an average page load time under 1 second, these two sites had page load times between 2 and 3 seconds:
Alexa's Ranking |
Site |
Total Links |
Crawler Runtime (seconds) |
Time per link (seconds) |
50 | Jcrew.com | 5,792 | 21,602.00 | 3.729627072 |
46 | Sky.com | 6,704 | 21,587.00 | 3.2200179 |
23 | Groupon.com | 7,363 | 21,596.00 | 2.933043596 |
10 | Ikea.com | 8,935 | 20,453.00 | 2.289087857 |
19 | Hm.com | 13,079 | 11,642.00 | 0.8901292148 |
13 | Cambridge.org | 24,977 | 21,603.00 | 0.8649157225 |
34 | Rebubble.com | 18,736 | 16,135.00 | 0.861176345 |
3 | Ebay.com | 16,010 | 10,790.00 | 0.6739537789 |
22 | BHPhotovideo.com | 22,315 | 12,368.00 | 0.5542460229 |
11 | Homedepot.com | 22,180 | 11,914.00 | 0.5371505861 |
Interestingly, if we were to re-rank the top e-commerce sites based on performance and working links, our ranking compared to Alexa's would look like this:
Alexa's |
mabl's |
Diffs |
Site |
41 | 1 | +40 | Bedbathandbeyond.com |
6 | 2 | +4 | Etsy.com |
45 | 3 | +42 | Ulta.com |
35 | 4 | +31 | Dickssportinggoods.com |
48 | 5 | +43 | Overstock.com |
38 | 6 | +32 | Zappos.com |
7 | 7 | 0 | Bestbuy.com |
21 | 8 | +13 | Humblebundle.com |
25 | 9 | +16 | Lowes.com |
3 | 10 | -7 | Ebay.com |
40 | 11 | +29 | Rei.com |
49 | 12 | +37 | Forever21.com |
47 | 13 | +34 | Cargurus.com |
29 | 14 | +15 | Shutterfly.com |
20 | 15 | +5 | Nordstrom.com |
34 | 16 | +18 | Rebubble.com |
39 | 17 | +22 | Walgreens.com |
30 | 18 | +12 | Sephora.com |
50 | 19 | +31 | Jcrew.com |
42 | 20 | +22 | Bodybuilding.com |
16 | 21 | -5 | Wiley.com |
19 | 22 | -3 | Hm.com |
9 | 23 | -14 | Target.com |
12 | 24 | -12 | Macys.com |
8 | 25 | -17 | Store.steampowered.com |
37 | 26 | +11 | Iherb.com |
13 | 27 | -14 | Cambridge.org |
46 | 28 | +18 | Sky.com |
32 | 29 | +3 | Oup.com |
15 | 30 | -15 | Nike.com |
2 | 31 | -29 | Netflix.com |
44 | 32 | +12 | Barnesandnoble.com |
17 | 33 | -16 | Costco.com |
11 | 34 | -23 | Homedepot.com |
1 | 35 | -34 | Amazon.com |
22 | 36 | -14 | BHPhotovideo.com |
4 | 37 | -33 | Amazon.co.uk |
23 | 38 | -15 | Groupon.com |
10 | 39 | -29 | Ikea.com |
You can see the full table showing all the sites' percentages of broken links here.
You can also see the full table of each site's average loading time here.
While most of these businesses have storefront locations, the experience of their online shops is paramount to their success as a company, as a growing portion of their revenues are coming from online sales. For smaller companies, a website may be the only place their users and customers can interact with them. This makes the identification of broken links and the assurance that a site can be used smoothly by all users a pressing matter for online shops.
How to test your website
Want to find out how your site measures up? You can create a free mabl trial account at app.mabl.com. The link crawler is run automatically when you add your test applications to mabl.
Feel free to also follow these steps from the mabl docs to add more link crawlers, add restrictions to the crawlers, and more!