Akamai Diversity

The Akamai Blog

Where does "State of the Internet" data come from?

Since early 2008, Akamai has been publishing the State of the Internet report each quarter - 14 editions in total through Q2 2011.  (The next edition, for Q3 2011, is currently being written and will be published in January 2012.)  As the editor, it has been very gratifying to see that the report series has been extremely well received, with regular coverage in the industry press (such as GigaOM, ZDNet, and eWeek) , and it has also positioned Akamai as a thought leader, affording us the opportunity to contribute to local and national broadband plans, both domestically and internationally.  The availability of high-speed Internet connectivity has become a key roadmap item for both telecoms and governments, with many now committing to achieving specific speeds and coverage levels over the next several years.
With each new report, I get questions about Akamai's data collection methodology, and why the data in our report differs from studies or data sets published by others, such as Pando, Ookla, or the OECD.  Suffice it to say that each study or data set has its own set of biases related to the underlying data collection methodology, Akamai included.
The Akamai Intelligent Platform sees on the order of one trillion requests for content each day from users in over 200 countries/regions around the world (238 in the second quarter, to be exact).  These content requests are serviced by over 100,000 servers deployed across more than 1,000 networks in 75 countries.  This immense amount of traffic provides Akamai with a massive data set that we are able to analyze to better understand connection speed trends (among others covered in the report).  For each content request, Akamai logs multiple pieces of data that are ultimately used for reporting, analytics, and billing.  Included within this data are three key pieces of information: the IP address associated with the request, the size of the file delivered in response to the request, and the amount of time it took to deliver that file.  From this, we can calculate a connection speed for that given request from that unique IP address.  Across the course of the quarter, Akamai collects thousands to millions of samples from each unique IP address.
Within the report, data is presented at country, U.S. state, and city levels - the geolocation of each unique IP address is done with Akamai's EdgeScape solution.  To calculate average connection speeds, an average is taken of all of the connection speeds calculated during the quarter from the unique IP addresses determined to be in a specific country/state/city.  To calculate average peak connection speeds, an average is taken of only the highest connection speed calculated from each unique IP address determined to be in a specific country/state/city.
Because of the broad global distribution of Akamai's platform, we believe that the measurements presented within the State of the Internet report are representative of end user experiences.  While there are factors beyond Akamai's control that can skew the connection speeds calculated by Akamai, such as contention for last mile bandwidth with non-Akamai content/applications (such as VoIP, gaming, or P2P), we are confident that the impact of these factors is mitigated by the tremendous number of measurements that are analyzed for the report.