Akamai Diversity

The Akamai Blog

The Flash Crash 4 Years Later: Ready for the Next One?

May 6, 2010 started like most days in the stock market. A few minutes before the U.S. equities markets opened at 9:30 AM, leading brokerage firms opened their internal "market open" conference calls. This is a common practice in the industry: get internal representatives from IT operations, networking, market data systems, software development, etc. on the phone together. Run through checklists making sure all systems are ready for the opening bell at 9:30 AM. Discuss the expected opening rush. What happened in the Asia and European markets overnight? How are the S&P futures looking? Do we have all our web servers up? Are we green? Anyone reporting yellow? OK everyone, continue monitoring your systems as the market opens at 9:30.
And then at about 9:25, the users start logging in. Some are there to submit buy and sell orders as close to the market open as possible. Others are there to watch the action live, like a sporting event. Whatever the reason, firms know that a large percentage of trading is done in those first few minutes, and they prepare every trading day for this.

But no one could have predicted the catastrophe that would unfold in the afternoon of May 6.

The Flash Crash:

Beginning at about 2:40 PM, with the DOW already down 300 points for the day, the stock market began to drop rapidly, loosing more than 600 points in five minutes. 20 minutes later, just after 3:00 PM, it had recovered the 600 point loss. The result looked like this:

Flash Crash Blog picture 1.png
Users quickly flooded the websites of all the online brokers. Prices on some individual stocks drop to just one cent per share for a short time before recovering. An astounding $1 Trillion in market value simply disappeared in that short time. And some brokerage websites failed.

Some websites experienced ten times their historical high visitor volumes on May 6, and despite the billions of dollars of investment in IT, and tremendous focus and effort on capacity planning and load testing, some sites could not handle the crush of the "flash crowd" than accompanied the "flash crash."

May 6, 2010 is still seared in the memory of those who lived it. I recently hosted the CIO from one of our stock exchange customers in our Cambridge headquarters. On a tour of our NOCC, we have historical trend lines showing spikes in our network delivery. "Which of those spikes is the Flash Crash?" he wanted to know. My answer was, "None of them. The Flash Crash was less that 1% of our overall SSL traffic delivery, and you won't see it on the chart."

How did Akamai handle the Flash Crash?

A number of the leading online brokers run their websites on Akamai. After the Flash Crash, I created the chart below, which shows the sum of all the traffic from all our brokerage customers. The Flash Crash is clearly noted on Thursday afternoon. What is also noted is the offload provided by Akamai to our brokerage customers during the crash, and it is this offload that helped keep our brokerage customers online. "Without Akamai, we would have gone down," was the comment I received from more that one of our customers.

Flash Crash Blog Picture 2.png
Will it happen again?

I believe it will happen again. I also believe that we are overdue. We came close with the AP Twitter Hack on April 23, 2013. Volatility is part of the system, and we can look back at events like Feb 27, 2007, the financial crisis of 2008, and the Flash Crash to realize that it is just a matter of time. Every 1-3 years something major occurs in the markets, and brokerage systems are pushed to their limits.

In the mean time, business is booming for many of the online brokers. TD Ameritrade, Charles Schwab, E*Trade and others are reporting record trading volumes.

I wish all the brokers continued success and welcome your comments.