Akamai Diversity

The Akamai Blog

Anomaly scoring is a better way to detect a real attack

The following is a guest post from Principal Enterprise Architect David Senecal and Principal Product Architect Ory Segal.


Internet security is constantly evolving and it's a challenge for all companies generating online revenue. Not only do they need to constantly reinvent themselves by adding more functionalities to allow their user to do more, but at the same time they need to protect their online transactions.

How to block a threat and not a real user?

One of the key problems in any security solution is how to handle false positives and false negatives - that is, how to avoid blocking valid users, while not missing malicious activity against the system. Web application firewalls (WAF) are no exception.

At Akamai, I have been working with the OWASP ModSecurity Core Rule Set for quite some time and to gain extensive mileage with the system, and the problem we've had with previous Core Rule Sets (CRS) was dealing with exactly this problem.

In some scenarios, a single rule firing on an HTTP request is often not deterministic enough to indicate a real attack. For example, finding the word "script" or "alert" independently in a request is not a good indication that a Cross Site Scripting attack is taking place.

However if you find both keywords together with some special markup characters in-between (something like "<script>alert("xss");</script>") malicious intent becomes more obvious.

Scoring 1.png

Improving the threat detection accuracy

In version 2.x of the CRS, OWASP introduced the concept of anomaly scoring as a better way to detect attacks more accurately. Each rule is built in such a way that it only holds one piece of the puzzle and is assigned a score. As a WAF parses a request through the multiple WAF rules that make up the CRS, it keeps track of the rules that fire and adds the score of each rule to compute the total anomaly score for a request. The WAF will then compare the request anomaly score with an inbound risk score rule threshold. If the score exceeded, the request is more likely to be malicious, otherwise the request is judged to be safe.

At a high level, the principle is simple, but to make it efficient there are some rules to follow:

  • Each rule in the rule set should look for specific keywords or patterns that are typical for an attack
  • Each rule cannot hold all the keywords typically used and found in an attack payload
  • Each rule must be given a score between 1 (informational) and 5 (critical). The score should then be assigned based on the risk

ModSecurity 2.x comes with 2 risk score rules: one that keeps track of all rules that fired during the request stage and another that adds to the score of the rules firing during the response stage. In practice, we discovered that it is very difficult, if not impossible, to find a single threshold that would work across the different types of attacks. The graph below shows the ideal threshold (highlighted in blue) for each type of attack.

Scoring 2.png

Akamai's Threat Research Team went back to the drawing board, and took this concept a step further, introducing attack specific risk score rules (Cross Site Scripting, SQL Injection, Command Injection, PHP Injection, HTTP Anomaly, Trojans and Remote File Include Attack). The result is the new Kona Rule Set that aims to reduce false positives and more accurately detect true attacks.

CRS 2.x in action

In order to put the new Kona Rule Set to the test, and do so by using proper methodology, Akamai's threat research team compared the accuracy of:

  • Akamai Kona Rules
  • A WAF policy running the CRS 1.6.1 ruleset with all rules in deny mode
  • A WAF policy running a standard 2.2.6 CRS rule (Vanilla OWASP CRS 2.2.6)

The testing process used both valid traffic (to measure false positives), as well as attack traffic (to measure false negatives).

We have been running an opt-in beta program with some of our customers to improve WAF accuracy for them.  As a result, we've been able to create a valid traffic sample that includes real world Internet traffic from some of the world's top 100 websites - including large amounts of real world traffic known to cause false positives. Attack traffic was also included from popular hacking tools, exploit tools, and web security scanners. These attack test cases represented 5% of the total sample set.

We consider the following measures:

  • Precision: % of blocked requests that were actual attacks
  • Recall: % of attacks that were actually blocked
  • Accuracy: % of decisions that were true
  • MCC*: Correlation between WAF decision and the actual nature of requests

* MCC is Matthews Correlation Coefficient: http://en.wikipedia.org/wiki/Matthews_correlation_coefficient

The table below shows the results of the experiment.

Scoring 3.png

Why should you use Anomaly Scoring?

The results clearly demonstrate that the policy running the Kona Rule Set blocked more real attacks than any other policy, and overall the Kona rule set is more in sync with reality and better able to detect actual attacks with a lower level of false positives.

It is worth mentioning that the measurements were done against an "out of the box" non-tuned configuration - specific WAF deployments are expected to have even better results using custom rules and more application-specific tuning.

Akamai Professional Services can help you to participate to the Kona Rule Set Beta program, we are always looking for customers to partner on our security research to improve our KONA security suite and reduce false positives even further.


David Senecal is Principal Enterprise Architect and Ory Segal is Principal Product Architect at Akamai.


Leave a comment