There's been a lot of talk about the recent very large DDoS attack against Spamhaus. Although I was quoted in some articles about it, I want to be clear that the attacks did not affect or involve Akamai or our customers. However, we have been the object of similar attacks in the past, and Akamai has a vested interest in making the Internet better - safer, more reliable and higher performing.
Unfortunately, there are some common... let's call them "misconfigurations" on the Internet which make these types of attacks both easier and much more destructive than they should be.
The one most talked about recently is DNS Amplification. This has been discussed many times in many places so I won't go into a great deal of detail. Very generally, a miscreant can send a query (a very small amount of data) to a DNS server and the server will send 100 or more times as much data in response to the victim. It is the equivalent of a stranger sending a postcard to a store and you get a gigantic catalog in return.
Now imagine getting 1000s or even millions of gigantic catalogs delivered to your house - per second. You couldn't even leave your house. What's more, if they send enough, the catalogs could jam your whole street or even neighborhood, causing a significant amount of collateral damage.
ISPs can stop this by ensuring their DNS servers only answer queries from users inside their own network. For example, if I run Patrick's Network, I would ignore any queries from users in Mike's Network. Answering those would be a waste of my resources, but more importantly, it can be used to abuse Mike's network. Even though locking down DNS servers is a good idea, many, many ISPs do not do this. There are currently upwards of 20 million DNS servers configured to answer queries from anyone on the Internet.
While this situation isn't ideal and ISPs should lock down their DNS servers, it is not the root cause of what the problem. The problem is actually source address spoofing. The reason the store sends you a catalog is because the return address on the postcard is yours, not the miscreant's. The store doesn't know you did not ask for the catalog - just the opposite, because yours is the return address, the store - or in the case of the Internet, the server - thinks it is doing you a favor by delivering the catalog.
Combining these two situations can create a couple of serious problems for the Internet. First, it allows a miscreant to get a massive "bang for their buck". A little attack traffic can be amplified 100X. Second, and in some ways more importantly, it hides the true source of the attack. When the victim analyzes the attack traffic, they only see the misconfigured DNS servers' addresses; they do not know the miscreant's address. Either of these reasons is sufficient to require stopping this, but the two combined can be disastrous.
The rules on the Internet (as much as the Internet has rules) state ISPs should check all the packets leaving their network to ensure the source addresses are from their own network. Put another way, ISPs are supposed to stop any postcards which have a fake return address. This is codified in something called a Best Common Practices document, BCP-38.
This problem is not new, and all major router vendors built simple ways to implement BCP-38 into their equipment years ago. However, many ISPs do not set the configuration, allowing spoofed packets to leave their network.
There are several reasons an ISP may not configure their network to disallow spoofing. Let's discuss a few of the most common:
* Lack of Knowledge
A lot of ISPs simply do not know they are supposed to do this, or understand the consequences of not doing it. This is why I am beating the drum, and working to get people to implement BCP-38. It's an important piece of Internet hygiene. To be clear, this is not a silver bullet. Configuring BCP-38 will not stop all attacks on the Internet, but it will help,
* Time & Effort
It takes time and effort to implement BCP-38. ISPs are businesses and they have a strong motive to be profitable. Implementing BCP-38 does not have clear profit or revenue behind it. A CFO may think the time an engineer spends implementing BCP-38 is time that could have been spent doing something to make money. I personally believe this is a false choice. Not configuring BCP-38 opens the ISP to abuse, and not just through DNS amplification. As important, we're dealing with shared fate - if no one does it, everyone is vulnerable; while if everyone does it, no one is vulnerable. A true accounting of the costs will likely show implementing BCP-38 is actually good for the bottom line.
* Risk to Revenue
Configuring filters means the risk of misconfiguring filters, which could in turn lead to filtering legitimate customer traffic. The larger a network, the more likely this scenario becomes. Businesses try hard not to upset their customers. Misconfiguration is a real threat and ISPs need to be careful of it. But everything an ISP does carries risk. This is why ISPs have process and procedure in place to help evaluate and mitigate these potential risks. In addition, it is possible to tie the outbound filters to inbound routing. In that case, the only way to break the outbound filter is to break the inbound routing, which means the customer would not be connected anyway.
In summary, while it may seem like there are good reasons not to implement BCP-38, there really are not. Yet there are many important reasons to implement it. Akamai strongly urges all ISPs to implement BCP-38 as quickly as possible.
Patrick Gilmore is a Chief Network Architect at Akamai