 |
DNS SURVEY: OCTOBER 2007
Goals
This study, commissioned by Infoblox, is designed to answer the following questions:
- How many nameservers are "out there", on the Internet, and which software they are running?
- Are people using IPv6 on their nameservers?
- Is anyone putting DNSSEC records in their zone data?
- How popular is SPF?
- How many authoritative nameservers advertise their software version?
- How many authoritative nameservers allow recursion?
- How many authoritative nameservers allow a zone transfer?
- Are authoritative nameservers topologically dispersed?
- Do delegations match authoritative NS records?
- Do all nameservers return the same TTL for NS records?
- Are SOA values within their suggested ranges?
- Do serial numbers for a zone match?
- How many zones have a lame server?
Datasets
Dataset I -- Random IP Addresses
For this dataset, we probed a sample of all IPv4 addresses in use on the Internet. Since we don't want to probe addresses that are not in use, we began with a snapshot of the global routing table taken from the Route Views project. Enumerating the advertised address space (and eliminating addresses that end with .0 and .255) results in 1,768,024,077 addresses. Next we randomly selected 5% of these for our survey, leaving us with 88,388,634 addresses to probe.
Dataset II -- Authoritative Nameservers
This dataset contains the nameservers authoritative for the zones in dataset III. We treat this as a separate dataset because here we are specifically interested in characteristics of authoritative nameservers, rather than zones or caching resolvers. Sometimes a nameserver IP address has many names. There are 368,333 distinct nameserver names in this set, and 251,898 distinct nameserver IP addresses.
Dataset III -- Second-Level Zones
This dataset is used to characterize the adoption of new technologies in the DNS, namely IPv6, DNSSEC, and SPF. To do so, we start with a sample of second-level domains in the COM and NET zones. We received a copy of the COM and NET zones from Verisign. Together they contain 83,889,164 second-level domains. We choose a random subset comprising 2,703,764 domains for this part of the survey.
Note that results based on this dataset are reported on a per-zone basis.
Results for Dataset I
Number of Nameservers
We sent a simple DNS query to each probe address in Dataset I. The query asks for the IPv4 address of a.root-servers.net, and has the RD bit set. A response indicates that there is a DNS server listening on the probe address. We did not implement timeouts and retransmission, so packet loss may affect these measurements. Queries and responses were logged with tcpdump. The following table summarizes the number of queries and replies:
| Hosts queried | 87,767,550 | |
| Hosts replying | 586,599 | 0.67% |
Note that some addresses sent back more than one reply, even though we sent only one query. This happens either because the probe address actually sends multiple responses for a single query, or because the reply source address may be different than the query destination address. We suspect that some ISPs intercept incoming DNS queries and forward them to a central server, but the server responds from its own address.
Based on these numbers we can estimate that there about 11,700,000 nameservers running on the Internet. Our previous estimate was 9,000,000 in August 2006.
Software Versions
For each nameserver found, we attempted to determine its software and version with two techniques. The first is to send a version.bind query to the address. The second is to use the fpdns tool to fingerprint the server.
The version.bind technique is simple because it is a single query/response. A BIND nameserver answers the query honestly unless the administrator has configured it to return a specific answer. Many people feel safer by obscuring the nameserver version string.
Since the version.bind answer cannot always be trusted, we also use fpdns to try to determine the software version. fpdns sends a number of different queries to a nameserver and uses known quirks and behavior to deduce the version. One downside is that fpdns can not always give specific answers. For example, it might say the software is "BIND 8.3.0-RC1 -- 8.4.4"
We give the fpdns result priority over version.bind if both return an answer. Otherwise, we assume the version.bind answer is correct if it looks like a version string. The following table shows the breakdown of common software and versions based on our analysis.
| BIND 9 | 249,484 | 64.53% |
| Nominum CNS/Embedded Linux* | 74,559 | 19.29% |
| PowerDNS | 25,469 | 6.59% |
| BIND 8 | 21,772 | 5.63% |
| Microsoft Windows DNS 2000 | 6,967 | 1.80% |
| Microsoft Windows DNS 2003 | 3,232 | 0.84% |
| BIND 4 | 856 | 0.22% |
| Cisco CNR | 604 | 0.16% |
| Microsoft Windows DNS NT4 | 394 | 0.10% |
| Other | 3,251 | 0.84% |
Note that 199,820 additional nameservers could not be identified. The percentages in the above table are based on the number of nameservers that could be identified.
*fpdns seems to have trouble telling the difference between certain embedded Linux devices and Nominum CNS.
Open Resolvers
To test for open recursors we send the nameserver a query for a name in a zone that we control. The query contains elements that uniquely identify the target nameserver. If our authoritative nameserver receives the forwarded query, we say that the target is an open resolver.
We found 805,508 open resolvers in this dataset. Extrapolating to the entire address space, we estimate that there are 16,000,000 open resolvers on the Internet.
Interestingly, we found more open resolvers, than "nameservers," in Dataset I. One reason is that we have different techniques for counting each kind. To find a nameserver we send a query for a.root-servers.net and look for a response. To count open resolvers, we send a query for a zone that we own and then look for the forwarded query at our authoritative nameserver. For reasons that we don't yet know, there are more hosts that will forward a DNS query than will reply to it. We are still investigating.
One interesting characteristic about the open resolvers is that many of them, 50% in fact, will forward our query to the authoritative nameserver, but will not forward the reply back to the client. This complicates our efforts to fingerprint these resolvers.
Another interesting characteristic is that the query hitting our authoritative nameserver does not come from the probe address. Most of the open resolvers first forward the query to a locally-configured caching resolver. Our authoritative nameserver received queries from only 70,783 distinct IP addresses.
The following table shows the software running on open resolvers. As seen below, we were unable to determine the software for most of the open resolver addresses:
| unknown | 465,450 | 57.8% |
| BIND 9 | 213,588 | 26.5% |
| Nominum CNS | 75,184 | 9.3% |
| PowerDNS | 25,242 | 3.1% |
| BIND 8 | 12,737 | 1.6% |
| Microsoft Windows DNS 2000 | 6,282 | 0.8% |
| Microsoft Windows DNS 2003 | 2,724 | 0.3% |
| sheerdns | 958 | 0.1% |
| dnsmasq-2 | 869 | 0.1% |
| Viking DNS module | 607 | 0.1% |
| BIND 4 | 538 | 0.1% |
| Cisco CNR | 533 | 0.1% |
| Microsoft Windows DNS NT4 | 360 | 0.0% |
Results for Dataset II -- Authoritative Nameservers
Software Versions
Here, we use the same fingerprinting technique as we did for dataset I. i.e., fpdns and then version.bind. The following table shows the software versions for authoritative nameservers:
| BIND 9 | 160,925 | 84.60% |
| BIND 8 | 12,521 | 6.58% |
| Microsoft Windows DNS 2000 | 8,741 | 4.60% |
| PowerDNS | 2,957 | 1.55% |
| Microsoft Windows DNS 2003 | 1,101 | 0.58% |
| BIND 4 | 690 | 0.36% |
| Microsoft Windows DNS NT4 | 514 | 0.27% |
| Nominum ANS | 182 | 0.10% |
| Cisco CNR | 90 | 0.05% |
| Other | 2,492 | 1.31% |
Note that 61,684 additional nameservers could not be identified. The percentages in the above table are based on the number of nameservers that could be identified. Additionally, the above table is based on distinct nameserver addresses rather than names. This eliminates biases caused by large domain owners that use different nameserver names, but a single nameserver address.
How many nameservers allow recursion?
We test recursion by querying our open resolvers database. The open resolver database tests addresses by sending queries for cryptographically generated names in the test.openresolvers.org domain. If our openresolvers.org authoritative nameserver receives a valid query, the tested address provides recursion.
The following table shows how many nameservers appear to recursively resolve domain names on our behalf:
| YES | 107,860 | 52.1% |
| NO | 117,380 | 47.9% |
| Undetermined | 26,658 |
Note, the above table is based on distinct nameserver addresses rather than names. This eliminates biases caused by large domain owners that use different nameserver names, but a single nameserver address.
How many nameservers allow a zone transfer?
We test for zone transfer with Net::DNS::Resolver's axfr_start and axfr_next functions. We say the nameserver allows the zone transfer if it returns at least one RR. Each nameserver is tested only once, even if it is authoritative for many zones.
The following table shows how many nameservers allowed us to transfer one of their zones:
| YES | 79,602 | 31.1% |
| NO | 176,420 | 68.9% |
And this table shows the software running on those nameservers that allow zone transfer:
| BIND 9 | 97,766 | 87.2% |
| BIND 8 | 5,071 | 4.5% |
| unknown | 4,939 | 4.4% |
| Microsoft Windows DNS 2000 | 2,788 | 2.5% |
| PowerDNS | 756 | 0.7% |
| BIND 4 | 342 | 0.3% |
| Microsoft Windows DNS NT4 | 175 | 0.2% |
| Nominum ANS | 132 | 0.1% |
| Microsoft Windows DNS 2003 | 90 | 0.1% |
Related to this measurement is the number of nameservers that allow TCP transactions at all. We test for TCP support by setting the `usevc' option to Net::DNS::Resolver and then making an SOA query for the zone. We say the nameserver supports TCP if it returns at least one answer RR. Each nameserver is tested only once. Here are the results:
| YES | 162,298 | 62.6% |
| NO | 97,171 | 37.4% |
Results for Dataset III - Zones
IPv6
In this part of the survey, we are interested in finding out how many zones have at least one nameserver with an IPv6 address. Even though Verisign (operator of the COM and NET registries) does not allow insertion of AAAA glue records in the COM/ZONE files, zone owners may elect to add them to their own zones.
To make the comparison, we ask for the IPv4 and IPv6 addresses (AAAA records) of all authoritative nameservers for each zone in Dataset III.
We found 2,158,777 zones in Dataset III with at least one IPv4-addressed nameserver. Among those, we also found 5,723 (0.27%) with at least one IPv6-addressed nameserver.
NOTE: Because the probed nameservers come from domains in the COM and NET zones, this data is not necessary an accurate reflection of worldwide IPv6 adoption for DNS nameservers.
DNSSEC
DNSSEC is an evolving protocol. The first version (RFC 2535, March 1999) defines the KEY, SIG, and NXT record types. The second version (RFC 4035, March 2005) essentially obsoletes the first-generation RR types and adds four new ones: DNSKEY, NSEC, RRSIG, and DS. We queried the set of nameservers for both old and new RR types.
Among the 2,053,150 zones with at least one working nameserver, we found 36 (0.0018%) with first-generation DNSSEC records.
We also found 8 zones publishing second-generation DNSSEC records. There is no overlap between the two first- and second-generation subsets.
Needless to say, DNSSEC adoption is still very small. Unfortunately, our use of the COM and NET zones probably under-represents DNSSEC adoption across the whole Internet. Some European CCTLDs have been more proactive in encouraging the use of DNSSEC.
SPF
SPF, which currently stands for Sender Policy Framework, is a way for organizations to list valid sources of email (i.e., IP addresses) for "from addresses" in their domain. Organizations that want to publish SPF information create a TXT record in their zone file. Mail transfer agents may lookup SPF information when receiving email messages. In addition to the TXT record format, SPF now also has its own DNS RR, type 99.
Among the 2,053,150 zones with at least one working nameserver, we found 257,924 (12.6%) with TXT-based SPF records, and 46 (0.0022%) that return the newer SPF (type 99) records.
Are a zone's nameservers topologically dispersed?
RFC 2182 states: secondary servers must be placed at both topologically and geographically dispersed locations on the Internet, to minimize the likelihood of a single failure disabling all of them. While we do not test for geographic diversity, we can easily examine the topologic aspects of a zone's nameservers. Our assumption that topologic locality implies geographic locality is is true in most, but not quite all cases. The following table shows the percentage of zones having all their authoritative nameservers within the given CIDR-sized subnets:
| CIDR Mask | Percentage | Cumulative |
|---|---|---|
| 32 | 4.7% | 4.7% |
| 31 | 7.3% | 12.0% |
| 30 | 2.4% | 14.3% |
| 29 | 3.9% | 18.2% |
| 28 | 1.5% | 19.8% |
| 27 | 1.9% | 21.7% |
| 26 | 0.7% | 22.4% |
| 25 | 0.7% | 23.1% |
| 24 | 0.7% | 23.8% |
| 23 | 3.9% | 27.6% |
| 22 | 7.2% | 34.8% |
| 21 | 1.3% | 36.1% |
| 20 | 1.6% | 37.8% |
| 19 | 6.4% | 44.1% |
| 18 | 1.1% | 45.2% |
| 17 | 6.1% | 51.3% |
| 16 | 0.1% | 51.4% |
Do delegations match authoritative NS records?
Here we examine whether delegation NS records (listed in the parent zone) match the authoritative NS records (found in the zone itself). We use the following six categories:
- Equal
The delegation records exactly match the authoritative records
- Disjoint
The delegation and authoritative records have no members in common
- Subset
The delegation records are a subset of the authoritative records
- Superset
The delegation records are a superset of the authoritative records
- Intersect
The delegation the authoritative records each contain members not found in the other set
- No Authoritative Nameservers
The supposedly-authoritative nameservers (learned from the parent zone) did not return any NS records for the zone
Note that we are only comparing nameserver names, rather than IP addresses. This may explain why there is a non-zero number of zones with Disjoint delegation/authoritative nameservers. The following table shows the percentage of each category:
| Equal | 1,212,202 | 47.9% |
| No Authoritative Nameservers | 1,048,908 | 41.4% |
| Disjoint | 98,178 | 3.9% |
| Subset | 95,776 | 3.8% |
| Superset | 53,072 | 2.1% |
| Intersect | 24,662 | 1.0% |
Do all nameservers return the same TTL for NS records?
Here we consider two slightly different cases. The first table shows the percentage of zones where the authoritative nameservers return NS records with mis-matched TTLs. This includes all NS records from all nameservers for a zone. The first column is the number of unique TTLs returned:
| 1 | 1,478,542 | 99.6% |
| 2 | 5,238 | 0.4% |
| 3 | 101 | 0.0% |
| 4 | 8 | 0.0% |
Similarly, the second table shows the percentage of nameservers that return differing NS record TTLs in response to a single query:
| 1 | 3,380,252 | 99.9% |
| 2 | 1,759 | 0.1% |
| 3 | 6 | 0.0% |
Are SOA values within their suggested ranges?
Refresh
RFC 1912 suggests that the SOA Refresh value should be at least 20 minutes and no longer than 12 hours. The following table shows the distribution of Refresh values:
| <20m | 58,428 | 2.8% |
| 20m to 12h | 1,727,908 | 84.2% |
| >12h | 266,814 | 13.0% |
Retry
RFC 1912 suggests that the SOA Retry value is typically less than the Refresh value. The following table shows the distribution of Retry values:
| retry <= refresh | 2,028,770 | 98.8% |
| retry > refresh | 24,380 | 1.2% |
Expire
RFC 1912 suggests that the SOA Expire value should be at least 2 weeks and no longer than 4 weeks. The following table shows the distribution of Expire values:
| <2w | 1,551,947 | 75.6% |
| 2w to 4w | 196,116 | 9.6% |
| >4w | 305,087 | 14.9% |
Minimum
RFC 2308 suggests that the SOA Minimum value (aka the negative caching TTL) should be at least 1 hour and no longer than 3 hours. Common nameserver implementations usually enforce their own negative caching limits of 3 hours or less. The following table shows the distribution of Minimum values:
| <1h | 390,839 | 19.0% |
| 1h to 3h | 462,718 | 22.5% |
| >3h | 1,199,593 | 58.4% |
Do serial numbers for a zone match?
| SAME | 1,956,222 | 95.3% |
| DIFF | 89,644 | 4.4% |
| OFFBYONE | 7,284 | 0.4% |
Note that this only includes zones that responded with at least one valid SOA record.
- SAME
all authoritative nameservers return the same serial number for the zone
- OFFBYONE
the minimum and maximum serial numbers among the set of authoritative nameservers differed by only one
- DIFF
one or more of the authoritative nameservers has a serial number different than the rest, where the difference is greater than one
How many zones have a lame server?
We test for lameness by sending an SOA query for a zone to an authoritative nameserver. If the response's RCODE is anything other than zero, we say the nameserver is lame. Here are the results:
| NOTLAME | 1,967,311 | 77.8% |
| LAME | 561,760 | 22.2% |
Note this only includes nameservers for which we have an IP address. If a zone has 3 nameservers and we have can't get an IP address for 1 nameserver, the zone is included in the analysis with the 2 good nameservers. In other words, an unresolvable nameserver is not counted as a lame server.
Notes
Nameservers Density
We wish to note that some nameservers are authoritative for many different zones. This may affect certain measurements, such as the topological dispersion. For example, if a company with many zones happens to have all its nameservers on the same subnet, that may artificially inflate the percentages in the "Are nameservers topologically dispersed?" section.
To get an idea of the how many zones each nameserver hosts, note that the mean number of zones per nameserver is 31.6 while the median is 2.0. The following table shows how many zones each of the top 25 nameservers are authoritative for:
| NS Rank | Zone Count | CDF |
|---|---|---|
| 1 | 364,561 | 5.88% |
| 2 | 364,165 | 11.76% |
| 3 | 363,387 | 17.63% |
| 4 | 363,298 | 23.49% |
| 5 | 362,413 | 29.34% |
| 6 | 166,900 | 32.04% |
| 7 | 16,4884 | 34.70% |
| 8 | 36,092 | 35.28% |
| 9 | 36,078 | 35.86% |
| 10 | 36,055 | 36.44% |
| 11 | 36,053 | 37.03% |
| 12 | 35,950 | 37.61% |
| 13 | 32,536 | 38.13% |
| 14 | 32,536 | 38.66% |
| 15 | 27,231 | 39.10% |
| 16 | 27,229 | 39.54% |
| 17 | 25,039 | 39.94% |
| 18 | 25,036 | 40.34% |
| 19 | 24,318 | 40.74% |
| 20 | 24,291 | 41.13% |
| 21 | 23,857 | 41.51% |
| 22 | 23,730 | 41.90% |
| 23 | 23,521 | 42.28% |
| 24 | 23,409 | 42.65% |
| 25 | 23,406 | 43.03% |
© 2020 The Measurement Factory.