Akamai Diversity
Home > Web Security

Recently in Web Security Category

Shellshock Update

The Shellshock vulnerability, originally announced as one critical issue in bash that allowed an adversary to execute arbitrary code, has grown from one vulnerability to six in the last week. For background on Shellshock, we've collected an overview and list of the vulnerabilities; for some history on Akamai's initial responses, read our original blog post
       
 Shellshock raised a lot of questions among our customers, peers, auditors, and prospects. This post addresses some of the most frequently asked questions, and provides an update on how Akamai is handling its operations during this industry-wide event.
 
Are Akamai production servers vulnerable?  What is the status of Akamai mitigation?
Akamai's HTTP and HTTPS edge servers never exposed any vulnerability to any of the six currently available CVEs, including the original ShellShock vulnerability.  Our SSH services (including NetStorage) were vulnerable post-authentication, but we quickly converted those to use alternate shells. Akamai did not use bash in processing end-user requests on almost any service. We did use bash in other applications that support our operations and customers, such as our report generation tools. We switched shells immediately on all applications that had operated via bash and are deploying a new version of bash that disables function exporting. 
Akamai's Director of Adversarial Resilience, Eric Kobrin, released a patch for bash that disables the Shellshock-vulnerable export_function field. His code has aggregated additional upstream patches as available, meaning that if you enable function import using his code, the same behaviors and protections available from the HEAD of the bash git tree are also available. His patch is available for public review, use, and critique.
We do not believe at this time that there is any customer or end user exposure on Akamai systems as a result of Shellshock.
What about Akamai's internal and non-production systems?
Akamai has a prioritized list of critical systems, integrated across production, testing, staging, and enterprise environments. Every identified critical system has had one or more of the following steps applied:
  • Verify that the system/application is not using bash (if so, we disabled the vulnerable feature in bash or switched shells);
  • Test that the disabled feature/new shell operates seamlessly with the application (if not, we repeated with alternate shells);
  • Accept upstream patches for all software/applications where available (this is an ongoing process, as vendors provide updates to their patches); and
  • Review/Audit system/application performance to update non-administrative access and disable non-critical functions.
Can we detect if someone has attempted to exploit ShellShock? Has Akamai been attacked?
Because the ShellShock Vulnerability is a Remote Code Execution vulnerability at the command shell, there are many possible exploits available using the ShellShock vulnerability. Customers behind our Web Application Firewall (WAF) can enable our new custom rules to prevent exploits using legacy CGI systems and other application-level exploits. These WAF rules protect against exploits of four of the six current vulnerabilities, all that apply to our customers' layer seven applications.
However, because ShellShock was likely present for decades in bash, we do not expect to be able to find definitive evidence -- or lack thereof -- of exploits.
There have been news reports indicating that Akamai was a target of a recent ShellShock-related BotNet attack. (See information about WopBot). Akamai did observe DDOS commands being sent to a IRC-controlled botnet to attack us, although the scale of the attack was insufficient to trigger an incident or need for remediation. Akamai was not compromised, nor were its customers inconvenienced.  We receive numerous attacks on a daily basis with little or no impact to our customers or the services we provide. 
Akamai's Cloud Security Research team has published an analysis of some attack traffic that Akamai has seen across its customers for Shellshock. As the authors note in that article, the kinds of payloads being delivered using the ShellShock vulnerability have been incredibly creative, with Akamai's researchers seeing more than 20,000 unique payloads. This creativity, coupled with the ease of the ShellShock vulnerability, is one of the many reasons that Akamai is keeping a close eye on all of the associated CVEs and continuing to update its systems and developing better protections for its customers, including custom WAF rules.
Where can I find updates on Akamai's WAF rules?
Information about our WAF rules can be found on our security site
How will Akamai communicate updates?
We will maintain this blog with interesting news from Akamai. 
As the list of CVEs and implications of ShellShock expand, we do our best to only deliver verified information, sacrificing frequency of updates for accuracy.
Akamai is maintaining additional materials for the public on its security site at , including a running tally of the bash-related vulnerabilities.
If you have questions that aren't addressed by one of these vehicles, please feel free to contact your account team.
In two weeks, I'll be at the Akamai Edge customer conference. It's a terrific opportunity to meet face-to-face with a lot of our customers and get their feedback on what's working for them and what we can improve upon. A robust Web Security track of talks is planned, and I'll be blogging about it. 

The security track will run each day of Edge. Here's a partial list of what's planned:

Environment Bashing

[UPDATE: 9/25/2014 11:30AM]

Akamai is aware that the fix to CVE-2014-6271 did not completely address the critical vulnerability in the Bourne Again Shell (bash). This deficiency is documented in CVE-2014-7169. The new vulnerability presents an unusually complex threat landscape as it is an industry-wide risk.

Akamai systems and internal Akamai control systems have been or are being urgently patched or otherwise mitigated in prioritized order of criticality.

Akamai has developed an emergency patch for today's vulnerability which makes function forwarding conditional on the compile-time switch "FUNCTION_EXPORT". We're using it for systems that can't be switched to the Almquist Shell. In the hope that it's useful to others, and for public review, we've posted that patch at https://bit.ly/ShellShockPatch.

_____________

 

About CVE-2014-6271
Today, CVE-2014-6271 was made public after its discovery last week by Stephane Chazelas. This vulnerability in bash allows an adversary who can pass commands to bash to execute arbitrary code.  As bash is a common shell for evaluating and executing commands from other programs, this vulnerability may affect many applications that evaluate user input, and call other applications via a shell.  This is not an Akamai-specific issue; this impacts any system that uses a vulnerable bash.
Akamai has validated the existence of the vulnerability in bash, and confirmed its presence in bash for an extended period of time.   We have also verified that this vulnerability is exposed in ssh---but only to authenticated sessions. Web applications like cgi-scripts may be vulnerable based on a number of factors; including calling other applications through a shell, or evaluating sections of code through a shell.  
There are several functional mitigations for this vulnerability: upgrading to a new version of bash, replacing bash with an alternate shell, limiting access to vulnerable services, or filtering inputs to vulnerable services.  Akamai has created a WAF rule to filter this exploit; see "For Web Applications" below for details.
Customer Mitigations
Systems under our customers' control which might be impacted include not only vulnerable web applications, but also servers which expose bash in various ways.  System owners should apply an updated bash with a fix for this vulnerability as expeditiously as possible.  In the meantime, there are several workarounds that may assist.
For SSH servers:  Evaluate the users who have access to critical systems, removing non-administrative users until the systems are patched.
For Web Applications: CGI functionality which makes calls to a shell can be disabled entirely as a short term measure; alternately WAF mitigations can be deployed.  The following Akamai WAF protections will assist customers that have purchased the Kona Web Application Firewall or Kona Site Defender services:
  • All customers of Akamai's web acceleration services (Ion, Dynamic Site Acceleration, EdgeSuite, Object Delivery, and similar) are protected against some attacks using this vulnerability by Akamai's HTTP normalization.  Attack text in Host headers, HTTP version numbers, and similar will be silently discarded by the Akamai Platform.  
  • Use of the "Site Shield" feature can further enhance the HTTP normalization protection by removing attackers' ability to connect directly to a customer web server.
  • Customers who use the new Kona Rule Set are protected against some attacks using this vulnerability if they enable the Command Injection risk scoring group.  This scoring group is being extended today to provide more comprehensive coverage by incorporating specific rule triggers for this vulnerability.  
  • We also have a custom WAF rule which will provide a targeted defense against this specific vulnerability.  This WAF rule operates by filtering on the four-byte attack string '() {' in the request header or body . It can be implemented to cover both headers and query strings (for maximum safety), or headers only (in the event that query strings generate too many false positives).  The custom rule is available to customers by contacting their account team.  It will be available for self service today.
  • This custom WAF rule will shortly be available for self-service provisioning for non-KRS customers.
Akamai Mitigations
Public-facing Akamai systems and internal Akamai control systems have been or are being urgently patched or otherwise mitigated in prioritized order of criticality. 
For many of our critical systems, Akamai first switched away from using bash to another shell, to protect those systems.  We did not make a universal switch, as the alternate shell was not completely feature-compatible with bash (not all of our systems would continue to operate with this change).
For other systems which could tolerate the downtime, we disabled those systems pending receiving an updated bash. For Akamai web applications which couldn't take an alternate shell and couldn't be disabled, we blocked many Akamai-owned CGI scripts at the Akamai Edge, until we developed and deployed the WAF rules above.
 
Do you have any evidence of system compromises?
 
No.  And unfortunately, this isn't "No, we have evidence that there were no compromises;" rather, "we don't have evidence that spans the lifetime of this vulnerability."  We doubt many people do - and this leaves system owners in the uncomfortable position of not knowing what, if any, compromises might have happened. 

PLXsert warns of Spike DDoS Toolkit

Akamai's Prolexic Security Engineering and Research Team (PLXsert) is tracking the spread of Spike, a new malware toolkit that poses a threat to embedded devices, as well as Linux and Windows systems.

Several versions of Spike can communicate and execute commands to infected Windows, desktop Linux and ARM-based devices running the Linux operating system (OS), PLXsert said in an advisory Wednesday morning.

Good Recognition for Akamai's Real-Time Web Monitor

Analyst Daniel Humphries has written a review of several threat monitoring tools for the "Software Advice" website, including a positive assessment of Akamai's Real-Time Web Monitor.

Ours was among five tools Humphries looked at in his report, "Spotlight: Threat Visualizations." The others were Kaspersky's Cyberthreat Real-Time Map, Digital Attack Map -- a joint project between Google and security vendor Arbor Networks -- the Deutsche Telekom Attack Meter, and Trend Micro's Global Spam Map. 

Humphries noted that threat visualization maps are becoming increasingly popular because of the "unique way in which they can illustrate cyber attacks," which are normally unseen to the human eye. "Both the educational and design value of these maps are crucial factors when it comes to successfully enlightening the public about the specific and global nature of security threats, so we wanted to find the best of the best, and highlight what it was that we liked most about each map," he said.

Akamai's Real-Time Web Monitor became a quick front-runner because it has the best of both worlds, he said: It shows a "very large and comprehensive range of threat data, while also having one of the simplest and cleanest interfaces of the maps we featured."

Read the full review here.

Akamai 1.png

Coming Soon: New Security Whiteboard Videos

Last year, we released a bunch of videos containing security whiteboard lessons on a variety of topics. This Thursday we shoot four new episodes. 

Below is a preview of each episode.

  • To see previous security whiteboard videos, go here and here.

Security Topics at Akamai Edge 2014: A Primer

Each year at Akamai Edge we update customers on some of the more persistent threats we've dealt with in the 12 months prior. Slides detailing the 2013 threat picture are available here. For an idea of what we'll be sharing at Edge 2014 in a couple weeks, I've assembled this primer. 

The following blog posts capture the main threats that have kept us busy in recent months:

Public Compliance Docs: The List So Far (Updated Sept. 18)

As previously noted, Akamai InfoSec has been working to make its most sought after compliance documents publicly available. The goal is to make it easier for customers to access the answers they regularly seek, and also to show potential new customers how we operate. 

We're building the foundation in the form of a compliance page on the Akamai Security microsite, and hope to publish up to two fresh public docs a month. What follows is a list of what we've done so far.

Web Vulnerabilities: Low-Hanging Fruit for DDoSers

A new Akamai PLXsert whitepaper was released this morning: "Web Vulnerabilities: The foundation of the most sophisticated DDoS campaigns." The paper can be downloaded here

Security practitioners know this much from long experience: 

Attackers who successfully build botnets and launch DDoS campaigns start by exploiting web vulnerabilities. It is the low-hanging fruit. In the white paper, PLXsert explores specific examples of the exploitation of popular web content management systems and web management suites and how these compromises have led to the development of some of the most advanced and difficult-to-stop DDoS campaigns.

suspect-ddos-attack.jpg

It's fitting that the Akamai Edge customer conference is in October. It's the same month as National Cyber Security Awareness Month, and we'll have a robust security track at Edge.