By Tim April
Update (Sept. 17, 2018): On Sunday, September 16th, the ICANN Board adopted a resolution instructing the ICANN Organization to continue the Root KSK Roll as planned, switching the Root KSK at 1600 UTC on October 11, 2018.
On October 11, 2018 -- for the first time ever -- the Root Key Signing Key (Root KSK), that is the single root of trust used to verify all DNSSEC responses, is scheduled to change. Validating resolvers that have not learned and started using the new key will view all answers as invalid, causing them to stop answering all queries, breaking user's Internet access.
Are you ready?
Introduction / What is DNSSEC
Domain Name System Security Extensions (DNSSEC) was introduced in 1997, with the publication of RFC2065. DNSSEC provides consumers of the Domain Name System (DNS) with a method by which responses can be verified using public key cryptography. Between 1997 and July 1, 2010, DNSSEC was employed by some DNS authority operators. However, at the time there was no single trust anchor, an authoritative entity, for which trust is assumed and not derived, that could be used to verify responses. On July 1, 2010, Internet Corporation for Assigned Names and Numbers (ICANN), Verisign, and the National Telecommunications and Information Administration (NTIA) completed the process of signing the root zone of the DNS. This made it possible for a single root of trust to be used to verify all DNSSEC responses, assuming each of the zones along the DNS chain were signed.
Since July 2010, the DNSSEC root of trust, also known as the Root Key Signing Key (Root KSK), has remained the same. Many of the keys used elsewhere in DNSSEC validation have rotated, similar to TLS certificates for websites, as is common with any public key system. Around the time that KSK-2010, the key currently in use, was introduced, the DNSSEC Practice Statement for the Root Zone KSK Operator document was created Internet Assigned Numbers Authority (IANA), the organization responsible for maintaining the root zone of the DNS. This document outlines DNSSEC operations of the root zone and calls for the rotation of the Root KSK after 5 years. Similarly, in 2013, the ICANN's Security and Stability Advisory Committee published an advisory calling for a roll of the Root KSK. The process of rolling the Root KSK should have started in 2015, but only began in October of 2016. The Root KSK generation process, resulting in the creation of KSK-2017, started the roll of the Root KSK and was originally planned to be completed in October of 2017.
There is an automated method for validating resolvers to learn and install the new Root KSK (RFC5011). However, not only do validating resolvers need to enable the update method, but the resolver also needs to be able to observe the key for a minimum of 30 days, and store that key in permanent storage. In certain situations, resolvers cannot write to permanent storage and cannot persist the new key. As mentioned, validating recursive resolvers not using the new key will stop answering queries, thus breaking user's Internet access. Because of the inability to reliably measure how many resolvers have not installed the new key, the decision was made by the ICANN Board to defer the rollout to allow more research and notification to be done. That deferment was for one year; and we are rapidly approaching that anniversary: October 11, 2018.
Many parties have weighed in on the plan to roll the Root KSK, including ICANN's Office of the CTO, the Security and Stability Advisory Committee via KSK Roll mailing list hosted by ICANN. The advice ranges from strong support of rolling the key this year to strong opposition. Both sides of this debate find themselves hoping for more data, outreach, and methods to help reduce the impact to end users. The ICANN board is considering the input they have received prior to the next board meeting where the Go / No-Go decision will be made, and communicated to the community sometime in the next week. We will update this post when the decision has been published.
What will happen
If the Root KSK roll goes as planned, the trust anchor Root KSK will switch from KSK-2010 to KSK-2017 starting October 11th, 2018. This will result in the responses being served from the root DNS nameservers, returning answers that are now chained to the new Root KSK. Because there is a 48 hour time to live (TTL) set in responses from the root name servers, some end users may see an impact some time between October 11th and October 13th. The timing of the impact depends primarily on how quickly the cache on their resolvers expire. Impacted end users will likely start out by experiencing host not found errors or other indications of DNS resolutions not working correctly, and the outage would appear to spread to other domains as more cache entries expire. For users experiencing an outage, using tools like ping, targeting an IP address rather than a domain name, can help rule out generic connectivity issues. For administrators, using IP addresses when accessing resources could be used as a temporary workaround for some applications until the resolution failures are resolved.
ICANN's Office of the CTO and others have estimated that the Root KSK Roll will result in less than a 1% denial of service (DoS) for Internet users as a result of this roll. These reports have acknowledged that there is a level of uncertainty involved due to a fundamental inability to measure some aspects of the DNS. Attempts have been made over the last two years to augment measurements, but the new data is not as conclusive as was desired.
How do I prepare for the Root KSK Roll?
If you are an end user, you will have to rely on your DNS service provider, often your ISP, or a public open resolver services (like Google DNS (220.127.116.11), Open DNS (18.104.22.168, 22.214.171.124), or Quad 9 (126.96.36.199)) to handle the key rotation correctly. Customers of Akamai's ETP or AnswerX hosted platforms and current versions of AnswerX (v2017.1 or later) or Nominum products with DNSSEC Validation enabled already have the new Root KSK trust anchor configured, so when the roll occurs, resolution should continue to work smoothly.
A first step of triaging if you may be impacted by the Root KSK roll, you can try to access http://dnssec-failed.org. If you are able to resolve and load that web page, the computer you used to test with is not configured with a validating resolver, and the Root KSK roll should have no impact on that system. Please note, this only verifies that the device the URL was access on is, or is not using DNSSEC validation, other devices such as Internet of Things (IoT), could be using different recursive resolvers without a simple way to test.
For those users who are using a validated resolver, there is an IETF Draft, commonly referred to as KSK Sentinel, which defines a set of specially crafted queries which end users can use to check if their resolver is prepared for the Root KSK roll. The website https://www.ksk-test.net was created to help end users test if your resolver validates DNSSEC, supports KSK Sentinel, and has the new trust anchor configured. KSK Sentinel is relatively new and has only been deployed to a small number of recursive resolvers. Your luck using this method may vary, but so far it is the only method available, aside from contacting your recursive resolver operator, to test that a recursive resolver is prepared for the Root KSK Roll.
If you are a recursive resolver operator, ICANN has provided pages that will walk you through checking and/or updating the trust anchors on a large number of resolver platforms. Operators may also want to consider checking if their platform supports RFC5011, a protocol which could automatically update and configure new trust anchors if implemented by your resolver. If this Root KSK roll continues as planned, implementing RFC5011 updates will not help, as the protocol requires 30 days to complete, thus September 10th, 2018 was the last day where the feature could be enabled.
Additionally, network operators should check their email for messages with the subject line, "Important DNS information about unprepared DNSSSEC-validating resolvers in ASXXXXX", where the XXXXX is replaced with your Autonomous System number(s). This message would be sent to the email address in the WHOIS for your IP addresses or ASNs. These emails provide some insight into resolvers that have appeared in logs at the root name servers with some level of uncertainty of their support for the new Root KSK. IP addresses showing up in these reports have been seen by some of the root name servers making queries that are often associated with validating resolvers that may not be prepared for the Root KSK roll. Administrators receiving these reports should use the procedures provided by ICANN for checking and/or updating their resolvers. Since clients could issue the same queries through non-validating recursive resolvers, there is a chance that false positives may appear.
In the case of the Root KSK roll, operators of authoritative name servers are at the mercy of the recursive resolvers used by clients, meaning resolution failures cannot be corrected by modifying the authoritative service. This includes nameserver services provided by Akamai such as Global Traffic Management (GTM) and Fast DNS (both primary and secondary DNS).
What about Akamai's DNS Services and Software?
Akamai offers many DNS products, solutions, and services. Customers of these services already have the new Root KSK trust anchor configured, so when the roll occurs, the resolution should continue to work smoothly. These services include AnswerX (Licensed 2017.1 or later, Cloud, & Managed), Enterprise Threat Protection (ETP), and DNSi (Nominum's CacheServe7).
Akamai's Authoritative DNS services, like Fast DNS and GTM, are unaffected by the Root KSK rollover. Unlike recursive resolvers, authoritative servers only answer zones for which they are configured, and have no need to interact with the Root KSK.
In addition, all of Akamai's platform uses DNS. Akamai engineers have confirmed that there will be no impact on the platform's ability to resolve domains after the planned Root KSK roll.
For customers with questions, please use the Akamai Community to get clarification from the appropriate Akamai DNS product or service.
Thank you to Warren Kumari for his assistance in preparing this post.