The following is a guest post from Principal Enterprise
Architect David Senecal and Principal Product Architect Ory Segal.
security is constantly evolving and it's a challenge for all companies
generating online revenue. Not only do they need to constantly reinvent
themselves by adding more functionalities to allow their user to do more, but
at the same time they need to protect their online transactions.
How to block a threat and not a real
One of the key problems in any security solution is how to handle false
positives and false negatives - that is, how to avoid blocking valid users,
while not missing malicious activity against the system.
Web application firewalls
(WAF) are no exception.
At Akamai, I have
been working with the OWASP ModSecurity Core Rule Set for quite some time and to
gain extensive mileage with the system, and the problem we've had with previous
Core Rule Sets (CRS) was dealing with exactly this problem.
In some scenarios,
a single rule firing on an HTTP request is often not deterministic enough to
indicate a real attack. For example, finding the word "script" or "alert" independently
in a request is not a good indication that a Cross Site Scripting attack is taking
However if you
find both keywords together with some special markup characters in-between (something
like "<script>alert("xss");</script>") malicious intent becomes
Improving the threat detection accuracy
In version 2.x
of the CRS, OWASP introduced the concept of anomaly scoring as a better way to
detect attacks more accurately. Each rule is built in such a way that it only
holds one piece of the puzzle and is assigned a score. As a WAF parses a
request through the multiple WAF rules that make up the CRS, it keeps track of
the rules that fire and adds the score of each rule to compute the total
anomaly score for a request. The WAF will then compare the request anomaly
score with an inbound risk score rule threshold. If the score exceeded, the
request is more likely to be malicious, otherwise the request is judged to be
At a high level,
the principle is simple, but to make it efficient there are some rules to
- Each rule in the
rule set should look for specific keywords or patterns that are typical for an
- Each rule cannot
hold all the keywords typically used and found in an attack payload
- Each rule must
be given a score between 1 (informational) and 5 (critical). The score should then
be assigned based on the risk
comes with 2 risk score rules: one that keeps track of all rules that fired
during the request stage and another that adds to the score of the rules firing
during the response stage. In practice, we discovered that it is very difficult,
if not impossible, to find a single threshold that would work across the
different types of attacks. The graph below shows the ideal threshold (highlighted
in blue) for each type of attack.
Akamai's Threat Research Team went back to the drawing board, and took this concept a step further, introducing attack specific risk score rules (Cross Site Scripting, SQL Injection, Command Injection, PHP Injection, HTTP Anomaly, Trojans and Remote File Include Attack). The result is the new Kona Rule Set that aims to reduce false positives and more accurately detect true attacks.
CRS 2.x in action
In order to put
the new Kona Rule Set to the test, and do so by using proper methodology,
Akamai's threat research team compared the accuracy of:
- Akamai Kona
- A WAF policy
running the CRS 1.6.1 ruleset with all rules in deny mode
- A WAF policy running
a standard 2.2.6 CRS rule (Vanilla OWASP CRS 2.2.6)
The testing process used both valid
traffic (to measure false positives), as well as attack traffic (to measure
We have been running an opt-in beta program
with some of our customers to improve WAF accuracy for them. As a result, we've been able to create a valid
traffic sample that includes real world Internet traffic from some of the world's
top 100 websites - including large amounts of real world traffic known to cause
false positives. Attack traffic was also included from popular hacking tools,
exploit tools, and web security scanners. These attack test cases represented 5%
of the total sample set.
We consider the
- Precision: % of
blocked requests that were actual attacks
- Recall: % of attacks that were actually blocked
- Accuracy: % of
decisions that were true
- MCC*: Correlation between WAF decision and
the actual nature of requests
* MCC is Matthews
Correlation Coefficient: http://en.wikipedia.org/wiki/Matthews_correlation_coefficient
The table below
shows the results of the experiment.
Why should you
use Anomaly Scoring?
The results clearly
demonstrate that the policy running the Kona Rule Set blocked more real attacks
than any other policy, and overall the Kona rule set is more in sync with
reality and better able to detect actual attacks with a lower level of false
It is worth
mentioning that the measurements were done against an "out of the box"
non-tuned configuration - specific WAF deployments are expected to have even
better results using custom rules and more application-specific tuning.
Professional Services can help you to participate to the Kona Rule Set Beta
program, we are always looking for customers to partner on our security
research to improve our KONA security
suite and reduce false positives even further.
David Senecal is Principal Enterprise Architect and Ory Segal is Principal Product Architect at Akamai.