Akamai Diversity

The Akamai Blog

Akamai University: Vulnerability Management vs. Pen Testing

Akamai Edge 2014 begins today and tomorrow with two days of Akamai University and API Boot camp. To coincide with this, I'm running two security lessons that are part of an upcoming video series. This is the first installment, written by Akamai CSIRT researcher Patrick Laverty.

Vulnerability Management vs. Pen Testing

Vulnerability assessment and pen testing both deal with finding and fixing security holes. But they are not the same thing.

Let's review the differences between the two, and how both are critical to the vulnerability management process.

First, let's start with some definitions.

  • A vulnerability assessment is the process of identifying, quantifying, and prioritizing (or ranking) the vulnerabilities in a system. Examples of systems for which vulnerability assessments are performed include, but are not limited to, information technology systems, energy supply systems, water supply systems, transportation systems, and communication systems. Such assessments may be conducted on behalf of a range of different organizations, from small businesses up to large regional infrastructures.
  • Vulnerability from the perspective of disaster management means assessing the threats from potential hazards to the population and to infrastructure. It may be conducted in the political, social, economic or environmental fields. Quite often in the web world, a vulnerability assessment is conducted by using a scanning tool. These tools are able to find many different types of vulnerabilities and can rate just how dangerous they are. When the scan is complete, the tool can offer a written report that can be reviewed and acted upon by the developers. The assessment will often not go any deeper than simply looking for the vulnerabilities, sort of like a checklist report back to the site owners.
  • A penetration test, or pentest, can have a great deal of overlap with a vulnerability assessment. In fact, a penetration tester will often perform a vulnerability assessment to some degree. The difference is that the pentester will then take things further and actually try to exploit a site, within the bounds of an agreed-upon test. Additionally, a pentester may string together multiple vulnerabilities to achieve the goal.

The vulnerability assessment will often just check to see where the problems likely are, but a good penetration test should show not only where they are but also how to actually exploit, or take advantage of them.

One way to think of this is if we see a ladder lying on the ground by itself as no big deal. We see an unlocked second-floor window as no big deal. 

But if someone is able to use the ladder to enter the second-floor window, this would become a successful attack.

A good penetration tester will not simply run a tool and submit the results, as tools can't find everything, but will go further and actually prove that the vulnerabilities exist, look for other vulnerabilities that a tool can't find and will use intelligence and logic to run multiple vulnerabilities in conjunction with each other.

But, both are used to assess weaknesses in a company's physical and IT systems.

Let's run through an example of each:

  • First, a vulnerability assessment: Most companies will frequently test their own security. At some companies it is ongoing, every minute of every day. All changes are immediately run through a vulnerability scanner.
  • Others may just check periodically. For example, here at Akamai, we have a service that runs every 24 hours that checks the strength of every employee's password. If the service is able to crack a password, the employee will be immediately notified to change it.
  • When this service runs, it is looking for or assessing a possible vulnerability -- weak passwords from employees.
  • A penetration test is conducted with someone who is experienced with breaking into systems. It will be conducted during an agreed-upon timeframe against a set of agreed-upon systems with agreed-upon information from the site owner. 
Generally there are three types of pentests:

  1. The "black box." This is the type that most people probably envision when they think of hacking. In a black box pentest, an attacker may only be given a single URL and no further information. 
  2. There is also a "gray box" test where the tester will be given additional information -- maybe what operating system is running or which database management system is being used. With this information, a tester can better focus the attack types and potentially achieve the goals faster.
  3. The last type of pentest is a "white box" test. This is where the attacker is given the keys to the kingdom and can have any information that they want including the ability to log in to any system with any level of access, can get diagrams of the infrastructure, and can have source code for review. Everything about the system is wide open for the tester's review.

You should be doing both: continually run a vulnerability assessment to find common issues and have developers fix them. 

At periodic intervals, whether annually or more frequently, hire a penetration tester to see how deep into your network and infrastructure they can go.

But most importantly, act on the results and fix the issues that they find. If the assessor or pentester can find vulnerabilities, you can be assured that the hackers will find them too.

That concludes our lesson for today.

1 Comment


I'm finishing my master's degree, working with support for "continuous" secure programming. The proof of concept of my technique is an Eclipse plug-in (https://marketplace.eclipse.org/content/early-security-vulnerability-detector-esvd/) that on compile time analyzes the source code (Java) to find security vulnerabilities.

I am performing evaluations on different techniques (variants/ combinations of pattern matching and data flow analysis), which are performed by different tools (ASIDE, LASPE+, CodePro and others) available for general use.

I would love to receive some feedback from you or your team on our first prototype.

I look forward to hearing from you.