Akamai Diversity
Home > Retail & Commerce > Black Friday and Cyber Monday RUM Insights - Part II

Black Friday and Cyber Monday RUM Insights - Part II

Performance across different device types

In Part 1 of this blog post we looked at traffic trends and performance on individual devices. When we compare performance across devices we see another interesting result. This is a large data set and it is made up of different pages from different websites so there is a large element of noise here, however all device types contain a substantial sample size and the randomness of the data is shared across each parameter.
01 - overall.JPG
If we focus in on the major holiday weekend we can see the two spikes in iPhone and Android as previously discussed but what's more interesting is the pattern of median performance. The experience of users on a Mac's is the best with a sub three seconds median page load time. Experiences on Windows devices are still respectable at just under four seconds. What is a surprise though is the positioning of the iPhone results hovering around the three seconds mark for page load. Next we have iPads jumping between five and six seconds and finally we have Android devices averaging out at about and eight seconds load time.

02 - overall zoomed.jpg
How can we explain these results?

First up let's take the iPad results. Slower than Mac's and slower than iPhones. There is no surprise that the Mac performs best, these are high-powered laptops that are normally connected to a wired or Wi-Fi Internet connection, and you would expect these to be the fastest. The iPhone result is great as it shows that you can provide good and similar experiences on mobile devices to the desktop counterpart, and with the use of good dedicated mobile sites (mdot's) these results reflect that. Retailers are more than likely to have a mobile strategy. When the mobile Internet becomes popular and with the continuing growth rates we see year over year with smart phone usage, it is impossible for retailers not to address this device type. A device with a small screen needs a smaller and more lightweight website to function well and more importantly to perform well. But what about iPads? Here we have a device that has a big screen; using a full size webpage on an iPad is easy, however this is a portable device that can be used over cellular connections and is also less powerful than a laptop in terms of processing power and memory. What strategy do you apply to this device? If you have a separate mobile and desktop site, you have already introduced extra work in managing and maintaining these. Adding a 3rd dedicated site only increases that burden. There is a growing trend of moving towards a responsive solution that addresses these issues, but if not implemented well it introduces performance issues. We see many sites where page size can often be the same across all device types, therefore decreasing performance on mobiles and tablets. Our CTO, Guy Podjarny, looks at RWD adoption but drills into page sizes on desktop and mobile devices here.

Looking at the two graphs below we can see there is little traffic served over cellular connections to iPads and when it is, it performs in a very similar way, meaning the device constraints are possible more of an issue than the network.

03 - ipad pie.jpg
04 - ipad line.jpg
There is clearly an opportunity here for retailers to serve tablet specific content to tablet users and improve this experience.

Does this result mean that retailers should ignore the network completely? We should also consider the "tethered" web view. This is someone using the Internet connection from his/her phone to access via a laptop. This does not represent a large volume of traffic but we have all been there in a coffee shop before an important meeting trying to put that one bit of useful information into our presentation.

Below shows the performance difference on Mac's over cellular and non-cellular connections.

05 - mac line.jpgThe graph below shows the performance gap between Windows based machines on cellular and non-cellular connections.

06 - windows line.jpg
The second major surprise in the results was the poor experience on Android devices. The first thing to consider is if Android users use cellular connections more than iPhone users, therefore giving iPhone users a performance advantage.

The two graphs below show the cellular vs. non-cellular breakdown between Android and iPhone devices. iPhone is actually slightly higher (34%) than Android (30%) so we cannot attribute poorer Android experiences to that.

07 - iphone pie.jpg
08 - android pie.jpg
Looking at all the data, the slower experience of users on Android devices was not caused by varying levels in connection type. What other characteristics of this user group could introduce such delays? One problem could be the diversity of device types, browsers and operating systems introduced by Android. A look into the data set showed over 8000 user agent strings, over 1400 different devices and nearly 50 major & minor Android versions.

The graph below shows the breakdown of the major Android version which shows the majority of users running on Kit Kat, 17% the previous version Jelly Bean and 9% on older versions. 4% could not be classified and 2% run the latest Lollipop version.

09 - android breakdown.jpg
Does this plethora of devices, browser versions and operating systems contribute towards the Android performance problem, or is it the devices themselves that are the problem? In a market where price is key, and last year's introduction of the $20 smartphone, is the market saturated with devices that aren't up-to-scratch for today's m-commerce audience? One thing is for sure, with this many combinations of variables, accurate "mobile" detection and making the right choice of content to send your end users is most difficult in this segment of data.

Summary

Analyzing the holiday data from last year gives great insight for retailers about when users will be shopping, browsing or researching on their site. It also provides a view into the types of device they use at different times of the day or season, possibly to carry out different types of task. The challenge of serving great experiences across a multitude of devices is currently a difficult problem, and unfortunately is only going to get worse. Introducing a strategy to tackle the multi device issue is the first step. The next step is to try and tailor content according to your user's situation that includes device, network connectivity, browser and location. Until all retailers have addressed these challenges, I will be doing my Holiday shopping on an iPhone and Mac!

Leave a comment