surveys

DNS SURVEY: OCTOBER 2007

Goals

This study, commissioned by Infoblox, is designed to answer the following questions:

  • How many nameservers are "out there", on the Internet, and which software they are running?
  • Are people using IPv6 on their nameservers?
  • Is anyone putting DNSSEC records in their zone data?
  • How popular is SPF?
  • How many authoritative nameservers advertise their software version?
  • How many authoritative nameservers allow recursion?
  • How many authoritative nameservers allow a zone transfer?
  • Are authoritative nameservers topologically dispersed?
  • Do delegations match authoritative NS records?
  • Do all nameservers return the same TTL for NS records?
  • Are SOA values within their suggested ranges?
  • Do serial numbers for a zone match?
  • How many zones have a lame server?

Datasets

Dataset I -- Random IP Addresses

For this dataset, we probed a sample of all IPv4 addresses in use on the Internet. Since we don't want to probe addresses that are not in use, we began with a snapshot of the global routing table taken from the Route Views project. Enumerating the advertised address space (and eliminating addresses that end with .0 and .255) results in 1,768,024,077 addresses. Next we randomly selected 5% of these for our survey, leaving us with 88,388,634 addresses to probe.

Dataset II -- Authoritative Nameservers

This dataset contains the nameservers authoritative for the zones in dataset III. We treat this as a separate dataset because here we are specifically interested in characteristics of authoritative nameservers, rather than zones or caching resolvers. Sometimes a nameserver IP address has many names. There are 368,333 distinct nameserver names in this set, and 251,898 distinct nameserver IP addresses.

Dataset III -- Second-Level Zones

This dataset is used to characterize the adoption of new technologies in the DNS, namely IPv6, DNSSEC, and SPF. To do so, we start with a sample of second-level domains in the COM and NET zones. We received a copy of the COM and NET zones from Verisign. Together they contain 83,889,164 second-level domains. We choose a random subset comprising 2,703,764 domains for this part of the survey.

Note that results based on this dataset are reported on a per-zone basis.

Results for Dataset I

Number of Nameservers

We sent a simple DNS query to each probe address in Dataset I. The query asks for the IPv4 address of a.root-servers.net, and has the RD bit set. A response indicates that there is a DNS server listening on the probe address. We did not implement timeouts and retransmission, so packet loss may affect these measurements. Queries and responses were logged with tcpdump. The following table summarizes the number of queries and replies:

Hosts queried 87,767,550  
Hosts replying 586,599 0.67%

Note that some addresses sent back more than one reply, even though we sent only one query. This happens either because the probe address actually sends multiple responses for a single query, or because the reply source address may be different than the query destination address. We suspect that some ISPs intercept incoming DNS queries and forward them to a central server, but the server responds from its own address.

Based on these numbers we can estimate that there about 11,700,000 nameservers running on the Internet. Our previous estimate was 9,000,000 in August 2006.

Software Versions

For each nameserver found, we attempted to determine its software and version with two techniques. The first is to send a version.bind query to the address. The second is to use the fpdns tool to fingerprint the server.

The version.bind technique is simple because it is a single query/response. A BIND nameserver answers the query honestly unless the administrator has configured it to return a specific answer. Many people feel safer by obscuring the nameserver version string.

Since the version.bind answer cannot always be trusted, we also use fpdns to try to determine the software version. fpdns sends a number of different queries to a nameserver and uses known quirks and behavior to deduce the version. One downside is that fpdns can not always give specific answers. For example, it might say the software is "BIND 8.3.0-RC1 -- 8.4.4"

We give the fpdns result priority over version.bind if both return an answer. Otherwise, we assume the version.bind answer is correct if it looks like a version string. The following table shows the breakdown of common software and versions based on our analysis.

BIND 9249,48464.53%
Nominum CNS/Embedded Linux*74,55919.29%
PowerDNS25,4696.59%
BIND 821,7725.63%
Microsoft Windows DNS 20006,9671.80%
Microsoft Windows DNS 20033,2320.84%
BIND 48560.22%
Cisco CNR6040.16%
Microsoft Windows DNS NT43940.10%
Other3,2510.84%

Note that 199,820 additional nameservers could not be identified. The percentages in the above table are based on the number of nameservers that could be identified.

*fpdns seems to have trouble telling the difference between certain embedded Linux devices and Nominum CNS.

Open Resolvers

To test for open recursors we send the nameserver a query for a name in a zone that we control. The query contains elements that uniquely identify the target nameserver. If our authoritative nameserver receives the forwarded query, we say that the target is an open resolver.

We found 805,508 open resolvers in this dataset. Extrapolating to the entire address space, we estimate that there are 16,000,000 open resolvers on the Internet.

Interestingly, we found more open resolvers, than "nameservers," in Dataset I. One reason is that we have different techniques for counting each kind. To find a nameserver we send a query for a.root-servers.net and look for a response. To count open resolvers, we send a query for a zone that we own and then look for the forwarded query at our authoritative nameserver. For reasons that we don't yet know, there are more hosts that will forward a DNS query than will reply to it. We are still investigating.

One interesting characteristic about the open resolvers is that many of them, 50% in fact, will forward our query to the authoritative nameserver, but will not forward the reply back to the client. This complicates our efforts to fingerprint these resolvers.

Another interesting characteristic is that the query hitting our authoritative nameserver does not come from the probe address. Most of the open resolvers first forward the query to a locally-configured caching resolver. Our authoritative nameserver received queries from only 70,783 distinct IP addresses.

The following table shows the software running on open resolvers. As seen below, we were unable to determine the software for most of the open resolver addresses:

unknown465,45057.8%
BIND 9213,58826.5%
Nominum CNS75,1849.3%
PowerDNS25,2423.1%
BIND 812,7371.6%
Microsoft Windows DNS 20006,2820.8%
Microsoft Windows DNS 20032,7240.3%
sheerdns9580.1%
dnsmasq-28690.1%
Viking DNS module6070.1%
BIND 45380.1%
Cisco CNR5330.1%
Microsoft Windows DNS NT43600.0%

Results for Dataset II -- Authoritative Nameservers

Software Versions

Here, we use the same fingerprinting technique as we did for dataset I. i.e., fpdns and then version.bind. The following table shows the software versions for authoritative nameservers:

BIND 9160,92584.60%
BIND 812,5216.58%
Microsoft Windows DNS 20008,7414.60%
PowerDNS2,9571.55%
Microsoft Windows DNS 20031,1010.58%
BIND 46900.36%
Microsoft Windows DNS NT45140.27%
Nominum ANS1820.10%
Cisco CNR900.05%
Other2,4921.31%

Note that 61,684 additional nameservers could not be identified. The percentages in the above table are based on the number of nameservers that could be identified. Additionally, the above table is based on distinct nameserver addresses rather than names. This eliminates biases caused by large domain owners that use different nameserver names, but a single nameserver address.

How many nameservers allow recursion?

We test recursion by querying our open resolvers database. The open resolver database tests addresses by sending queries for cryptographically generated names in the test.openresolvers.org domain. If our openresolvers.org authoritative nameserver receives a valid query, the tested address provides recursion.

The following table shows how many nameservers appear to recursively resolve domain names on our behalf:

YES107,86052.1%
NO117,38047.9%
Undetermined26,658 

Note, the above table is based on distinct nameserver addresses rather than names. This eliminates biases caused by large domain owners that use different nameserver names, but a single nameserver address.

How many nameservers allow a zone transfer?

We test for zone transfer with Net::DNS::Resolver's axfr_start and axfr_next functions. We say the nameserver allows the zone transfer if it returns at least one RR. Each nameserver is tested only once, even if it is authoritative for many zones.

The following table shows how many nameservers allowed us to transfer one of their zones:

YES79,60231.1%
NO176,42068.9%

And this table shows the software running on those nameservers that allow zone transfer:

BIND 997,76687.2%
BIND 85,0714.5%
unknown4,9394.4%
Microsoft Windows DNS 20002,7882.5%
PowerDNS7560.7%
BIND 43420.3%
Microsoft Windows DNS NT41750.2%
Nominum ANS1320.1%
Microsoft Windows DNS 2003900.1%

Related to this measurement is the number of nameservers that allow TCP transactions at all. We test for TCP support by setting the `usevc' option to Net::DNS::Resolver and then making an SOA query for the zone. We say the nameserver supports TCP if it returns at least one answer RR. Each nameserver is tested only once. Here are the results:

YES162,29862.6%
NO97,17137.4%

Results for Dataset III - Zones

IPv6

In this part of the survey, we are interested in finding out how many zones have at least one nameserver with an IPv6 address. Even though Verisign (operator of the COM and NET registries) does not allow insertion of AAAA glue records in the COM/ZONE files, zone owners may elect to add them to their own zones.

To make the comparison, we ask for the IPv4 and IPv6 addresses (AAAA records) of all authoritative nameservers for each zone in Dataset III.

We found 2,158,777 zones in Dataset III with at least one IPv4-addressed nameserver. Among those, we also found 5,723 (0.27%) with at least one IPv6-addressed nameserver.

NOTE: Because the probed nameservers come from domains in the COM and NET zones, this data is not necessary an accurate reflection of worldwide IPv6 adoption for DNS nameservers.

DNSSEC

DNSSEC is an evolving protocol. The first version (RFC 2535, March 1999) defines the KEY, SIG, and NXT record types. The second version (RFC 4035, March 2005) essentially obsoletes the first-generation RR types and adds four new ones: DNSKEY, NSEC, RRSIG, and DS. We queried the set of nameservers for both old and new RR types.

Among the 2,053,150 zones with at least one working nameserver, we found 36 (0.0018%) with first-generation DNSSEC records.

We also found 8 zones publishing second-generation DNSSEC records. There is no overlap between the two first- and second-generation subsets.

Needless to say, DNSSEC adoption is still very small. Unfortunately, our use of the COM and NET zones probably under-represents DNSSEC adoption across the whole Internet. Some European CCTLDs have been more proactive in encouraging the use of DNSSEC.

SPF

SPF, which currently stands for Sender Policy Framework, is a way for organizations to list valid sources of email (i.e., IP addresses) for "from addresses" in their domain. Organizations that want to publish SPF information create a TXT record in their zone file. Mail transfer agents may lookup SPF information when receiving email messages. In addition to the TXT record format, SPF now also has its own DNS RR, type 99.

Among the 2,053,150 zones with at least one working nameserver, we found 257,924 (12.6%) with TXT-based SPF records, and 46 (0.0022%) that return the newer SPF (type 99) records.

Are a zone's nameservers topologically dispersed?

RFC 2182 states: secondary servers must be placed at both topologically and geographically dispersed locations on the Internet, to minimize the likelihood of a single failure disabling all of them. While we do not test for geographic diversity, we can easily examine the topologic aspects of a zone's nameservers. Our assumption that topologic locality implies geographic locality is is true in most, but not quite all cases. The following table shows the percentage of zones having all their authoritative nameservers within the given CIDR-sized subnets:

CIDR MaskPercentageCumulative
324.7%4.7%
317.3%12.0%
302.4%14.3%
293.9%18.2%
281.5%19.8%
271.9%21.7%
260.7%22.4%
250.7%23.1%
240.7%23.8%
233.9%27.6%
227.2%34.8%
211.3%36.1%
201.6%37.8%
196.4%44.1%
181.1%45.2%
176.1%51.3%
160.1%51.4%

Do delegations match authoritative NS records?

Here we examine whether delegation NS records (listed in the parent zone) match the authoritative NS records (found in the zone itself). We use the following six categories:

Equal

The delegation records exactly match the authoritative records

Disjoint

The delegation and authoritative records have no members in common

Subset

The delegation records are a subset of the authoritative records

Superset

The delegation records are a superset of the authoritative records

Intersect

The delegation the authoritative records each contain members not found in the other set

No Authoritative Nameservers

The supposedly-authoritative nameservers (learned from the parent zone) did not return any NS records for the zone

Note that we are only comparing nameserver names, rather than IP addresses. This may explain why there is a non-zero number of zones with Disjoint delegation/authoritative nameservers. The following table shows the percentage of each category:

Equal1,212,20247.9%
No Authoritative Nameservers1,048,90841.4%
Disjoint98,1783.9%
Subset95,7763.8%
Superset53,0722.1%
Intersect24,6621.0%

Do all nameservers return the same TTL for NS records?

Here we consider two slightly different cases. The first table shows the percentage of zones where the authoritative nameservers return NS records with mis-matched TTLs. This includes all NS records from all nameservers for a zone. The first column is the number of unique TTLs returned:

11,478,54299.6%
25,2380.4%
31010.0%
480.0%

Similarly, the second table shows the percentage of nameservers that return differing NS record TTLs in response to a single query:

13,380,25299.9%
21,7590.1%
360.0%

Are SOA values within their suggested ranges?

Refresh

RFC 1912 suggests that the SOA Refresh value should be at least 20 minutes and no longer than 12 hours. The following table shows the distribution of Refresh values:

<20m58,4282.8%
20m to 12h1,727,90884.2%
>12h266,81413.0%

Retry

RFC 1912 suggests that the SOA Retry value is typically less than the Refresh value. The following table shows the distribution of Retry values:

retry <= refresh2,028,77098.8%
retry > refresh24,3801.2%

Expire

RFC 1912 suggests that the SOA Expire value should be at least 2 weeks and no longer than 4 weeks. The following table shows the distribution of Expire values:

<2w1,551,94775.6%
2w to 4w196,1169.6%
>4w305,08714.9%

Minimum

RFC 2308 suggests that the SOA Minimum value (aka the negative caching TTL) should be at least 1 hour and no longer than 3 hours. Common nameserver implementations usually enforce their own negative caching limits of 3 hours or less. The following table shows the distribution of Minimum values:

<1h390,83919.0%
1h to 3h462,71822.5%
>3h1,199,59358.4%

Do serial numbers for a zone match?

SAME1,956,22295.3%
DIFF89,6444.4%
OFFBYONE7,2840.4%

Note that this only includes zones that responded with at least one valid SOA record.

SAME

all authoritative nameservers return the same serial number for the zone

OFFBYONE

the minimum and maximum serial numbers among the set of authoritative nameservers differed by only one

DIFF

one or more of the authoritative nameservers has a serial number different than the rest, where the difference is greater than one

How many zones have a lame server?

We test for lameness by sending an SOA query for a zone to an authoritative nameserver. If the response's RCODE is anything other than zero, we say the nameserver is lame. Here are the results:

NOTLAME1,967,31177.8%
LAME561,76022.2%

Note this only includes nameservers for which we have an IP address. If a zone has 3 nameservers and we have can't get an IP address for 1 nameserver, the zone is included in the analysis with the 2 good nameservers. In other words, an unresolvable nameserver is not counted as a lame server.

Notes

Nameservers Density

We wish to note that some nameservers are authoritative for many different zones. This may affect certain measurements, such as the topological dispersion. For example, if a company with many zones happens to have all its nameservers on the same subnet, that may artificially inflate the percentages in the "Are nameservers topologically dispersed?" section.

To get an idea of the how many zones each nameserver hosts, note that the mean number of zones per nameserver is 31.6 while the median is 2.0. The following table shows how many zones each of the top 25 nameservers are authoritative for:

NS RankZone CountCDF
1364,5615.88%
2364,16511.76%
3363,38717.63%
4363,29823.49%
5362,41329.34%
6166,90032.04%
716,488434.70%
836,09235.28%
936,07835.86%
1036,05536.44%
1136,05337.03%
1235,95037.61%
1332,53638.13%
1432,53638.66%
1527,23139.10%
1627,22939.54%
1725,03939.94%
1825,03640.34%
1924,31840.74%
2024,29141.13%
2123,85741.51%
2223,73041.90%
2323,52142.28%
2423,40942.65%
2523,40643.03%

© 2013 The Measurement Factory.