Akamai Diversity
Home > Bill Brenner

Recently by Bill Brenner

One of the challenges of working in the security community is that you are expected to be fully aware of risk at all times. But as humans we all slip up sometimes. I was reminded of that yesterday when I helped out with a training session for new Akamai employees.

In these training sessions, we go over Akamai security procedures and how employees are to conduct themselves. There are the obvious technological best practices, like locking a computer when leaving the desk, choosing strong passwords and not sharing sensitive company data with outsiders by email and the various forms of social media. Physical security is also covered -- where to go if the building is evacuated, what not to say about the company in the crowded restaurants, coffee shops and sidewalks that surround our headquarters, and so on.

Also see: 

As we went down the checklist, I could think of at least two cases where adversaries got the better of me, despite all my experience. Before I go into examples, I should point out that these are mistakes I made before joining Akamai.

On the physical side, my mouth has gotten me into trouble. I have a deep, loud voice that can be heard from much more of a distance than I thought until it was pointed out the hard way. I was covering a court story and sitting in a press room, talking to a colleague about what I was working on. I thought we were alone as it was early in the morning. But someone who worked for a competing publication heard me from across a room separated by curtains and banks of computers. I was working on a scoop, and the competitor, hearing me talk about it, chased the story and got it published before I could publish mine.

It taught me that you shouldn't talk about your company's initiatives in a crowded Starbucks, in a city or town full of people who work for your competitors. You've likely heard the saying "Loose lips sink ships." Loose lips can also land your company's intellectual property in the hands of competitors. 

In the training session, we also talked about how to tell if someone is attempting to dupe you into downloading malware with a phishing attack. We know the danger signs -- emails and other messages made to look like they come from legitimate sources, telling us to click a link to fix some glitch with a bill, order or something else the victim is bound to care about. I've written hundreds of stories about it, yet a couple years ago I fell for the oldest trick in the book.

It came in as a direct message on Twitter from a colleague who sat in the next cube over from me at the office. He's a nice, mild-mannered chap, so when I got a tweet in his name, I opened the link without thought. Well, that's actually not true. I did have thoughts --based on his tweet: "Hello somebody is saying very bad rumors about you... (URL removed)"

I've been in this profession for a long time, and have found myself on the receiving end of blistering criticism plenty of times. It's a simple byproduct of the job. And yet I had to know who was spreading bad rumors about me. And I had to know right that second. I clicked the link and got a slow-loading site that ended in a request for my Twitter username and password. Another huge red flag. But someone was out there spreading rumors about me, you see, and I had to know what it was. So I plugged in my credentials.

As the screen of my Android froze up, I got the sinking feeling that I had just committed an act of supreme dumbness. By then, it was too late.

Soon after that, a friend on Twitter sent me this message:

"Guessing you didn't mean to post that..."

It turns out the bad guys started using my Twitter account to send out a variety of spam messages to friends, including the one I fell for.

I changed all my passwords for everything, and the Twitter madness ceased.

It goes to show that we can never be too careful, and that we must always be vigilant.


Telling Akamai's Security Story: Part 2

Three months ago when I started at Akamai, I told you the goal was to tell some Akamai InfoSec stories and make it clear how A.) we make sure our own house is secure, and B.) we provide an ironclad defense for customers. Here's an update to explain how we're doing that.

There are the almost-daily posts in this blog. There's plenty going on in our security department every day, which means there's never a shortage of topics to type up. To distribute those posts and add additional insight from the team, we've created Akamai InfoSec pages on Facebook, Twitter, Google+ and LinkedIn. In addition to my own posts, CSO Andy Ellis has posted some important updates on the BREACH vulnerability, environmental controls at Planetary Scale, and DNS reflection defense

Meanwhile, others from Akamai InfoSec have stepped forward to contribute blog posts, most notably Dave Lewis, Christian Ternus and Meg Grady-Troia. We've released a few videos as well, featuring Andy Ellis, CSIRT Director Michael Smith and Security Intelligence Director Joshua Corman.

Now comes the next phase: The launch this month of The Akamai Security Podcast. I've spent the last week setting up the recording and editing equipment, and will begin with some introductory interviews of Akamai InfoSec team members. The podcasts will launch weekly.

Finally, we're making progress developing a security page on the Akamai website where you'll be able to access all the above content as well as slideshows, infographics, research papers and articles on topics that matter to customers and the security community as a whole.

If you don't see what you're looking for along the way, please let us know.

See you online.

Bill Brenner 
Senior Program Manager, Editorial
Akamai InfoSec

SEA Attacks Illustrate Need for Better DNS Security

The Syrian Electronic Army (SEA) -- a pro-Assad hacking group -- is making misery for some of the biggest entities on the Internet.

The SEA's activities have attracted plenty of media attention this week. Users couldn't access many high-profile websites Tuesday after SEA launched a targeted phishing attack against a reseller for Melbourne IT, an Australian domain registrar and IT services company. According to the IDG News Service, the attack allowed hackers to change the DNS records for several domain names including nytimes.com, sharethis.com, huffingtonpost.co.uk, twitter.co.uk and twimg.com -- a domain owned by Twitter.

"This resulted in traffic to those websites being temporarily redirected to a server under the attackers' control," the news service reported. "Hackers also made changes to the registration information for some of the targeted domains, including Twitter.com. However, Twitter.com itself was not impacted by the DNS hijacking attack."

Akamai InfoSec's CSIRT team has been monitoring the attacks. From our perspective, recent events illustrate the need for better DNS security and better awareness of spear phishing, a favorite tactic of the SEA.

Michael Kun, a security response engineer on Akamai InfoSec's CSIRT team, told me companies should be getting more serious about registry locks so the bad guys can't tamper with DNS servers. 

Domain owners can and should ask their registrars to put the registry locks in place -- something Melbourne IT did for nytimes.com and the other sites. The lock is deployed at the registry level -- with companies that administer such domain extensions as .net, .org and .com.

Kun said companies should also seek out registrars that require two-factor authentication and pressure other registrars to support two-factor authentication as well.

"Unfortunately, the problem is really with the registrars, so there's not much that customers can do directly except to vote with their dollars," Kun said.


Security Ethics and The Hacker Academy

If you work outside the security community, the word "hacker" is often misunderstood. A hacker is seen as someone who operates outside the law, troublemakers who are only in the business of engineering attacks and causing chaos. Because of that misconception, I often feel the need to educate the masses.

To that end, I'd like to direct you to the blog of security company Tripwire, which has a talented team we often collaborate with. Its latest post is about what's known as The Hacker Academy, a three-year-old online ethical hacker training program designed by penetration testers to help educate young security practitioners with hands-on training.

My good friend Anthony Freed spoke with Joseph Sokoly (@jsokoly), a vulnerability engineer at MAD Security who I've written about in this blog before.

Sokoly is a rising star in the security community, and does a lot of work with The Hacker Academy. The blog post includes a video interview where he discusses the San Francisco-based academy, whose online resources are available around the clock.

Please check it out.


Akamai FedRAMP Compliance is Huge for Security

Yesterday was a big day around here. We achieved Federal Risk and Authorization Management Program (FedRAMP) compliance as a cloud services provider. 

Big deal, you say? Why, yes. It is. 

FedRAMP is a U.S. government-wide program that standardizes the approach to security assessment, authorization, and continuous monitoring for cloud products and services. Specifically, Akamai's globally distributed, publicly shared cloud services platform has received "Provisional Authority to Operate (P-ATO)" from the FedRAMP Joint Authorization Board (JAB). 

As Akamai Public Sector VP Tom Ruff noted, "Achieving FedRAMP compliance allows public sector organizations to trust the Akamai Intelligent Platform as the foundation for their cloud computing projects, while at the same time supporting their defense-in-depth strategies. As important, FedRAMP compliance is another example of Akamai's commitment to serving the public sector and complements our DNSSEC, IPv6 and HIPAA compliant offerings, currently supporting nearly all Cabinet-level agencies."

Akamai CSO Andy Ellis said on Twitter: "The FedRAMP accreditation for @Akamai covers pretty much our entire commercial service portfolio."

The U.S. General Services Administration lists the following goals and benefits of FedRAMP on its website:

--Accelerate the adoption of secure cloud solutions through reuse of assessments and authorizations
--Increase confidence in security of cloud solutions
--Achieve consistent security authorizations using a baseline set of agreed upon standards to be used for Cloud product approval in or outside of FedRAMP
--Ensure consistent application of existing security practices
--Increase confidence in security assessments
--Increase automation and near real-time data for continuous monitoring

--Increases re-use of existing security assessments across agencies
--Saves significant cost, time and resources - "do once, use many times"
--Improves real-time security visibility
--Provides a uniform approach to risk-based management
--Enhances transparency between government and cloud service providers (CSPs)
--Improves the trustworthiness, reliability, consistency, and quality of the Federal security authorization process

The Akamai InfoSec compliance and public sector staffs worked long and hard to reach this moment. For me, it's one of many examples of how dedicated people here are to making Akamai products and services secure. They were tireless and tenacious in reaching this point, and I'm honored to share the same workspace with them.


DDoS Attacks: China's Weekend of Irony

I can't help but see irony in all the news reports this morning about China suffering one of the worst DDoS attacks it has ever seen. China is usually seen as the place where attacks begin, a perception bolstered by findings in Akamai's most recent "State of The Internet" report

Of all the reports on the weekend DDoS against China, this passage from The Wall Street Journal's article explains things best, in my opinion:

The attack, which was aimed at the registry that allows users to access sites with the extension ".cn," likely shut down the registry for about two to four hours, according to CloudFlare, a company that provides Web performance and security services for more than a million websites. Though the registry was down, many service providers store a record of parts of the registry for a set period of time, meaning that the outage only affected a portion of websites for some users.

The article quotes CloudFlare Chief Executive Matthew Prince, who said the company observed a 32-percent drop in traffic for the thousands of Chinese domains on the company's network during the attack compared with the same time 24 hours earlier. The article also notes that while China is among the best there is at carrying out attacks, it's in a much weaker position to deal with attacks that come its way. From the report:

China has one of the most sophisticated filtering systems in the world and analysts rate highly the government's ability to carry out cyber attacks. Despite this, China is not capable of defending itself from an attack, which CloudFlare says could have been carried out by a single individual.

Our most recent "State of The Internet" report fingered China as the country from which most attack traffic originated:

During the first quarter of 2013, Akamai observed attack traffic originating from 177 unique countries/regions, consistent with the count in the prior quarter. China remained the top source of observed attack traffic, 
though its percentage declined by nearly a fifth from the prior quarter. This decline is likely related to Indonesia making a sudden appearance in the second place slot, after a 30x increase quarter-over-quarter.

China topped the list in the previous "State of the Internet" report as well. At the time, SecurityWeek reported:

The fact that China remained at the top of the list isn't so surprising. Earlier this year, Mandiant released a hefty report outlining evidence its researchers had gathered linking an "overwhelming" number of cyber-attacks to China, even to a specific military group. Even the Verizon's 2013 Data Breach Investigation Report called out China for cyber-espionage and other targeted attacks. Verizon claimed China was behind 30 percent of data breaches in its report. "Looking at the full year, China has clearly had the most variability (and growth) across the top countries/regions, originating approximately 16 [percent] of observed attack traffic during the first half of 2012, doubling into the third quarter, and growing further in the fourth quarter," Akamai said.

Below is a chart from our latest report on countries that produce the most attack traffic.


Mapping Networks and Data: Safety in Numbers

Last week I wrote about how redundancy of systems is an important part of Akamai's security at Planetary Scale. This post focuses on another way we keep Internet traffic flowing smoothly in the face of attempted attacks: network and data mapping.

Mapping isn't a security technique in itself. Every big network can be mapped out. But there is certainly a huge security benefit to it. In Akamai's case, we've mapped out every server deployed around the globe. If one goes down for any reason, we can quickly reroute traffic to other servers because we know exactly where everything is. 
In my research, I've found some good writing on how Akamai maps the Internet. One example is a blog post called "Intelligent User Mapping in the Cloud," written by Eugene Zhang, a senior enterprise architect with Akamai's Professional Services organization. The other is a report called "How Akamai Maps the Net: An Industry Perspective," written by George Economou.
Economou wrote in his 2010 paper:
The dynamic nature of Akamai's scalable and flexible distributed systems design relies heavily on, and benefits greatly from, the rigorous efforts invested in network mapping. Akamai's notion of network mapping is relatively broad, and is crafted into several specific methods for real-time service operation or longterm data analysis. Akamai's network presence and access to traffic provides a very unique vantage point to understand the Internet and how it is operating; these examples provide a sampling of how Akamai takes advantage of this information for very specific purposes. Whatever shapes the Internet morphs into in the future, you can bet that Akamai will be present and will have new ways of mapping it.
Doing so seems complex when you consider the size of the operation. As of 2010, he noted, we had over 60,000 servers deployed in about 1,400 data centers on about 900 networks worldwide. Geographically, these data centers were in about 650 cities in 76 countries around the world. 
I look at this as a case study in the concept of safety in numbers. If you walk around dangerous neighborhoods in a big city by yourself, you're going to be defenseless against attackers waiting around the corner. If you have other people with you, you become a much tougher target and are more likely to be left alone. 
In the case of the Internet, there's safety in numbers for the technology deployed to route traffic. If we only had a few servers deployed in a couple countries, it would be much easier to do serious damage to the flow of Internet traffic. But our technology is so spread out and numerous that the traffic is unstoppable.
That's especially the case because of our mapping process. If one guy goes down in a fight, we know exactly where the reinforcements are and can deploy then quickly.

DDoS Attacks Used As Cover For Other Crimes

Protecting customers from DDoS attacks is an Akamai InfoSec specialty. When we see DDoS attempts against our customers, the typical thinking is that someone is doing it to force sites into downtime, which can cost a business millions in lost online sales. 

But sometimes, these attacks are simply a cover operation to distract the victim while something else is going on. 

A story that caught our attention in SC Magazine and elsewhere drives home the point. The article, published Wednesday, explains how the bad guys have stolen millions from U.S. banks while distracting the victims with DDoS activity. From the article:

Criminals have recently hijacked the wire payment switch at several US banks to steal millions from accounts, a security analyst says. Gartner vice president Avivah Litan said at least three banks were struck in the past few months using "low-powered" distributed denial-of-service (DDoS) attacks meant to divert the attention and resources of banks away from fraudulent wire transfers simultaneously occurring. The loses "added up to millions [lost] across the three banks," she said. "It was a stealth, low-powered DDoS attack, meaning it wasn't something that knocked their website down for hours."

The story has gotten the attention of other publications as well. From CNet's article on the subject:

Security researchers have previously highlighted the growing trend of using DDoS attacks to hide fraudulent activity at banks. Dell SecureWorks Counter Threat Unit issued a report (PDF) in April that warned that a popular DDoS toolkit called Dirt Jumper was being used to divert bank employees' attention from attempted fraudulent wire transfers of up to $2.1 million.

Though Litan's write-up on the Gartner website has generated a lot of fresh attention, these kinds of attacks aren't all that new. Nearly a year ago, the threat was outlined in a joint paper from the FBI, Financial Services Information Sharing and Analysis Center (FS-ISAC) and the Internet Crime Complaint Center (IC3). The Sept. 17, 2012 alert said, among other things:

Recent FBI reporting indicates a new trend in which cyber criminal actors are using spam and phishing e-mails, keystroke loggers, and Remote Access Trojans (RAT) to compromise financial institution networks and obtain employee login credentials. The stolen credentials were used to initiate unauthorized wire transfers overseas. The wire transfer amounts have varied between $400,000 and $900,000, and, in at least one case, the actor(s) raised the wire transfer limit on the customer's account to allow for a larger transfer. In most of the identified wire transfer failures, the actor(s) were only unsuccessful because they entered the intended account information incorrectly.

Litan offered some additional advice:

"One rule that banks should institute is to slow down the money transfer system while under a DDoS attack," she wrote. "More generally, a layered fraud prevention and security approach is warranted."

Below: This graphic, from the latest Akamai State of the Internet report, shows which sectors are most impacted by DDoS attacks. 

Figure 05 Q1 2013.jpg

Microsoft's August Patch Matrix

Microsoft released it's monthly patch load this week. To help identify and deploy the security fixes, here's a table showing the different bulletins, the severity of the flaws, and the products impacted.

Bulletin IDBulletin Title and Executive SummaryMaximum Severity Rating and Vulnerability ImpactRestart RequirementAffected Software
MS13-059Cumulative Security Update for Internet Explorer (2862772

This security update resolves eleven privately reported vulnerabilities in Internet Explorer. The most severe vulnerabilities could allow remote code execution if a user views a specially crafted webpage using Internet Explorer. An attacker who successfully exploited the most severe of these vulnerabilities could gain the same user rights as the current user. Users whose accounts are configured to have fewer user rights on the system could be less impacted than users who operate with administrative user rights.
Remote Code Execution
Requires restartMicrosoft Windows, 
Internet Explorer
MS13-060Vulnerability in Unicode Scripts Processor Could Allow Remote Code Execution (2850869)

This security update resolves a privately reported vulnerability in the Unicode Scripts Processor included in Microsoft Windows. The vulnerability could allow remote code execution if a user viewed a specially crafted document or webpage with an application that supports embedded OpenType fonts. An attacker who successfully exploited this vulnerability could gain the same user rights as the current user. Users whose accounts are configured to have fewer user rights on the system could be less impacted than users who operate with administrative user rights.
Remote Code Execution
May require restartMicrosoft Windows
MS13-061Vulnerabilities in Microsoft Exchange Server Could Allow Remote Code Execution (2876063)

This security update resolves three publicly disclosed vulnerabilities in Microsoft Exchange Server. The vulnerabilities exist in the WebReady Document Viewing and Data Loss Prevention features of Microsoft Exchange Server. The vulnerabilities could allow remote code execution in the security context of the transcoding service on the Exchange server if a user previews a specially crafted file using Outlook Web App (OWA). The transcoding service in Exchange that is used for WebReady Document Viewing uses the credentials of the LocalService account. The Data Loss Prevention feature hosts code that could allow remote code execution in the security context of the Filtering Management service if a specially crafted message is received by the Exchange server. The Filtering Management service in Exchange uses the credentials of the LocalService account. The LocalService account has minimum privileges on the local system and presents anonymous credentials on the network.
Remote Code Execution
May require restartMicrosoft Server Software
MS13-062Vulnerability in Remote Procedure Call Could Allow Elevation of Privilege (2849470)

This security update resolves a privately reported vulnerability in Microsoft Windows. The vulnerability could allow elevation of privilege if an attacker sends a specially crafted RPC request.
Elevation of Privilege
Requires restartMicrosoft Windows
MS13-063Vulnerabilities in Windows Kernel Could Allow Elevation of Privilege (2859537)

This security update resolves one publicly disclosed vulnerability and three privately reported vulnerabilities in Microsoft Windows. The most severe vulnerabilities could allow elevation of privilege if an attacker logged on locally and ran a specially crafted application. An attacker must have valid logon credentials and be able to log on locally to exploit these vulnerabilities. The vulnerabilities could not be exploited remotely or by anonymous users.
Elevation of Privilege
Requires restartMicrosoft Windows
MS13-064Vulnerability in Windows NAT Driver Could Allow Denial of Service (2849568)

This security update resolves a privately reported vulnerability in the Windows NAT Driver in Microsoft Windows. The vulnerability could allow denial of service if an attacker sends a specially crafted ICMP packet to a target server that is running the Windows NAT Driver service.
Denial of Service
Requires restartMicrosoft Windows
MS13-065Vulnerability in ICMPv6 could allow Denial of Service (2868623)

This security update resolves a privately reported vulnerability in Microsoft Windows. The vulnerability could allow a denial of service if the attacker sends a specially crafted ICMP packet to the target system.
Denial of Service
Requires restartMicrosoft Windows
MS13-066Vulnerability in Active Directory Federation Services Could Allow Information Disclosure (2873872) 

This security update resolves a privately reported vulnerability in Active Directory Federation Services (AD FS). The vulnerability could reveal information pertaining to the service account used by AD FS. An attacker could then attempt logons from outside the corporate network, which would result in account lockout of the service account used by AD FS if an account lockout policy has been configured. This would result in denial of service for all applications relying on the AD FS instance.
Information Disclosure
May require restartMicrosoft Windows

2003 Blackout: An Early Lesson in Planetary Scale?

On the drive to work this morning, I listened to a report about this being the 10th anniversary of the massive blackout that plunged an area from New York City to Toronto into darkness. I immediately thought of a post Akamai CSO Andy Ellis wrote recently called "Environmental Controls at Planetary Scale."

It might be overreaching to say the 2003 blackout was an early case study in the success and failures of controls at Planetary Scale. Andy was talking about the environmental controls in data centers around the world. The blackout wasn't something individual data centers had much control over, and the power failure was geographically limited to a section of the U.S. and Canada. The blackout's root cause was a software glitch in an alarm system inside one of FirstEnergy Corp.'s control rooms in Ohio. Workers apparently didn't realize they needed to redistribute power after overburdened transmission lines collapsed onto overgrown trees. A manageable local blackout thus snowballed into widespread electric grid failure.

Still, I can't help but think of the parallels. Andy's blog post examined the pros and cons of investing large sums of money in data center environmental controls. He wrote: 

Is the cost worth the hassle? If you run one data center, then the costs might worthwhile - after all, it's only a few capital systems, and a few basis point improvements in MTBCF will likely be worth that hassle (both in operational false positives as well as deployment cost). But what if you operate in thousands of data centers, most of them someone else's?  The cost multiplies significantly, but the marginal benefit significantly decreases - as any given data center improvement only affects such a small portion of your systems.  Each data center in a planetary scale environment is now as critical to availability as a power strip is to a single data center location.  Mustering an argument to monitor every power strip would be challenging; a better approach is to have a drawer full of power strips, and replace ones that fail.

I see lessons here in how we manage interconnected electrical systems where a failure in one place can spill over to many other places the world over. Security experts have said and written much in recent years about the threat to global power grids. Among other things, they've warned, a hacker could compromise SCADA controls in one power station and maximize the damage if the target is the weak link in a much bigger chain of power distribution centers.

The ways in which we manage the threat carry similar pros and cons to that of the environmental control management Andy wrote about.

On this particular anniversary, I throw it out there as food for thought.