    |
  |
DNS SURVEY: AUGUST 2006
Goals
This study, commissioned by Infoblox, is designed to answer the following questions:
- How many nameservers are "out there", on the Internet, and which software they are running?
- Are people using IPv6 on their nameservers?
- Is anyone putting DNSSEC records in their zone data?
- How popular is SPF?
- How many authoritative nameservers advertise their software version?
- How many authoritative nameservers allow recursion?
- How many authoritative nameservers allow a zone transfer?
- Are authoritative nameservers topologically dispersed?
- Do delegations match authoritative NS records?
- Do all nameservers return the same TTL for NS records?
- Are SOA values within their suggested ranges?
- Do serial numbers for a zone match?
- How many zones have a lame server?
Datasets
Dataset I -- Random IP Addresses
For this dataset, we probed a sample of all IPv4 addresses in use on the Internet. Since we don't want to probe addresses that are not in use, we began with a snapshot of the global routing table taken from the Route Views project. Enumerating the advertised address space (and eliminating addresses that end with .0 and .255) results in 1,561,642,207 addresses. Next we randomly selected 5% of these for our survey, leaving us with 78,088,096 addresses to probe.
Dataset II -- Second-Level Zones
This dataset is used to characterize the adoption of new technologies in the DNS, namely IPv6, DNSSEC, and SPF. To do so, we start with a sample of second-level domains in the COM and NET zones. We received a copy of the COM and NET zones from Verisign. Together they contain 62,713,978 second-level domains. We choose a random subset of just under 2,000,000 domains for this part of the survey.
Note that results based on this dataset are reported on a per-zone basis.
Dataset III -- Authoritative Nameservers
This dataset contains 347,714 authoritative nameservers for 1,857,393 COM and NET zones. It is different than Dataset II because here we are interested in nameserver characteristics, rather than zone characteristics.
Number of Nameservers
We sent a simple DNS query to each probe address in Dataset I. The query asks for the IPv4 address of a.root-servers.net. A response indicates that there is a DNS server listening on the probe address. We did not implement timeouts and retransmission, and our query has the Recursion Desired bit cleared. Queries and responses were logged with tcpdump. The following table summarizes the number of queries and replies:
| Queries sent out | 78,088,096 | |
| Replies received | 476,079 | 0.61% |
| Addresses sending replies | 455,743 | 0.58% |
| Repeated replies | 20,336 | 0.026% |
Note that some addresses sent back more than one reply, even though we sent only one query. This happens either because the probe address actually sends multiple responses for a single query, or because the reply source address may be different than the query destination address. We suspect that some ISPs intercept incoming DNS queries and forward them to a central server, but the server responds from its own address.
Based on these numbers we can estimate that there about 9,000,000 nameservers running on the Internet. Our previous estimate (based on this technique) was 7,500,000 in April 2005.
Software Versions
For each address that sent a reply, we attempted to determine its software and version with two techniques. The first is to send a version.bind query to the address. The second is to use the fpdns tool to fingerprint the server.
The version.bind technique is simple because it is a single query/response. A BIND nameserver answers the query honestly unless the administrator has configured it to return a specific answer. Many people feel safer by obscuring the nameserver version string.
Since the version.bind answer cannot always be trusted, we also use fpdns to try to determine the software version. fpdns sends a number of different queries to a nameserver and uses known quirks and behavior to deduce the version. One downside is that fpdns can not always give specific answers. For example, it might say the software is "BIND 8.3.0-RC1 -- 8.4.4"
We give the fpdns result priority over version.bind if both return an answer. Otherwise, we assume the version.bind answer is correct if it looks like a version string. The following table shows the breakdown of common software and versions based on our analysis. You can also see the full table with all software versions found.
| BIND 9 | 201,723 | 60.74% |
| Embedded Linux* | 51,720 | 15.57% |
| BIND 8 | 45,547 | 13.71% |
| PowerDNS | 14,448 | 4.35% |
| Microsoft Windows DNS 2000 | 11,548 | 3.48% |
| Microsoft Windows DNS 2003 | 3,246 | 0.98% |
| BIND 4 | 1,387 | 0.42% |
| Microsoft Windows DNS NT4 | 868 | 0.26% |
| Cisco CNR | 254 | 0.08% |
| Other | 1,369 | 0.41% |
Note that 122,188 additional nameservers could not be identified. The percentages in the above table are based on the number of nameservers that could be identified.
*fpdns seems to have trouble telling the difference between certain embedded Linux devices and Nominum CNS.
IPv6
In this part of the survey, we are interested in finding out how many zones have at least one nameserver with an IPv6 address. Even though Verisign (operator of the COM and NET registries) does not allow insertion of AAAA glue records in the COM/ZONE files, zone owners may elect to add them to their own zones.
To make the comparison, we begin by counting the number zones with at least one IPv4 nameserver address. Some of the zones in Dataset II may have expired, or may be ``parked'' so that even though the domain is registered, it is not really in use. The following table shows how we arrived at the number of zones with an IPv4 nameserver address:
| zones in the set | 1,983,762 |
| zones still listed in the parent zone | 1,873,695 |
| zones with a working nameserver | 1,756,827 |
| zones with at least one RR | 1,745,801 |
| zones with an SOA RR | 1,390,127 |
| zones with an "NS A" record | 976,098 |
Why is "NS A" the count so much lower than "at least one RR" and "one working nameserver?" Probably because the second level zone maintainers don't correctly list their own nameservers.
To find a "good nameserver," we:
- get list of referrals from the TLD
- loop over the referral list and return the first nameserver that sends at least one record in response to ANY, A, and SOA queries.
To count the number of nameservers with A records, we:
- Ask the one-good-NS for the zone SOA record
- look for NS RRs in the authority section of the response
- query a local caching resolver for the A record of each NS
Compared to the 976,098 zones with IPv4 nameserver addresses, we also found 1,864 that have IPv6 addresses (AAAA records). In other words, about 0.2% of the second-level zones in COM and NET are using IPv6 addresses for their nameservers.
NOTE: Because the probed nameservers come from domains in the COM and NET zones, this data is not necessary an accurate reflection of worldwide IPv6 adoption for DNS nameservers.
DNSSEC
DNSSEC is an evolving protocol. The first version (RFC 2535, March 1999) defines the KEY, SIG, and NXT record types. The second version (RFC 4035, March 2005) essentially obsoletes the first-generation RR types and adds four new ones: DNSKEY, NSEC, RRSIG, and DS. We queried the set of nameservers for both old and new RR types.
Among the 1,756,827 zones with at least one working nameserver, we found 16 (0.001%) with first-generation DNSSEC records.
Coincidentally, we also found 16 zones publishing second-generation DNSSEC records. There is no overlap between the two first- and second-generation subsets.
Needless to say, DNSSEC adoption is still very small. Unfortunately, our use of the COM and NET zones probably under-represents DNSSEC adoption across the whole Internet. Some European CCTLDs have been more proactive in encouraging the use of DNSSEC.
SPF
SPF, which currently stands for Sender Policy Framework, is a way for organizations to list valid sources of email (i.e., IP addresses) for "from addresses" in their domain. Organizations that want to publish SPF information create a TXT record in their zone file. Mail transfer agents may lookup SPF information when receiving email messages. In addition to the TXT record format, SPF now also has its own DNS RR, type 99.
Among the 1,756,827 zones with at least one working nameserver, we found 87,859 (5%) with TXT-based SPF records. We did not find any of the new SPF (type 99) records.
How many authoritative nameservers openly advertise their software version?
Note, this is similar to the measurements taken on Dataset I, above. However, here we are only probing authoritative nameservers. Also, we did not run fpdns here. Instead, we only send version.bind queries to each authoritative nameserver just once.
About 38% of nameservers do not openly advertise their software version. Most of those that do are running BIND software:
| NORMAL_BIND | 176178 | 50.7% |
| NOANSWER | 88400 | 25.4% |
| TIMEOUT | 42984 | 12.4% |
| OTHER | 31774 | 9.1% |
| EMPTY | 3295 | 0.9% |
| POWERDNS | 3245 | 0.9% |
| SEMI_NORMAL_BIND | 1588 | 0.5% |
| NSD | 179 | 0.1% |
| ULTRADNS | 38 | 0.0% |
| TINYDNS | 28 | 0.0% |
| NORRS | 4 | 0.0% |
- NORMAL_BIND
the version string matches the pattern /^[489]\.[0-9]/
- SEMI_NORMAL_BIND
the version string matches the pattern /^{named,bind} [489]\.[0-9]/
- TIMEOUT
we did not receive a DNS reply to the query
- NOANSWER
we received a reply, but it did not contain any answer RRs
- EMPTY
the version.bind reply was the empty string
How many nameservers allow recursion?
We test recursion by querying our open resolvers database. The open resolver database tests addresses by sending queries for cryptographically generated names in the test.openresolvers.org domain. If our openresolvers.org authoritative nameserver receives a valid query, the tested address provides recursion.
The following table shows how many nameservers appear to recursively resolve domain names on our behalf:
| YES | 119912 | 52.6% |
| NO | 108125 | 47.4% |
| NOADDRESS | 1 |
Note, the above table is based on distinct nameserver addresses rather than names. This eliminates biases caused by large domain owners that use different nameserver names, but a single nameserver address.
How many nameservers allow a zone transfer?
We test for zone transfer with Net::DNS::Resolver's axfr_start and axfr_next functions. We say the nameserver allows the zone transfer if it returns at least one RR. Each nameserver is tested only once, even if it is authoritative for many zones.
The following table shows how many nameservers allowed us to transfer one of their zones:
| YES | 101947 | 29.3% |
| NO | 245766 | 70.7% |
Related to this measurement is the number of nameservers that allow TCP transactions at all. We test for TCP support by setting the `usevc' option to Net::DNS::Resolver and then making an SOA query for the zone. We say the nameserver supports TCP if it returns at least one answer RR. Each nameserver is tested only once. Here are the results:
| YES | 204863 | 58.9% |
| NO | 142850 | 41.1% |
Are nameservers topologically dispersed?
RFC 2182 states: secondary servers must be placed at both topologically and geographically dispersed locations on the Internet, to minimize the likelihood of a single failure disabling all of them. While we do not test for geographic diversity, we can easily examine the topologic aspects of a zone's nameservers. Our assumption that topologic locality implies geographic locality is is true in most, but not quite all cases. The following table shows the percentage of zones having all their authoritative nameservers within the given CIDR-sized subnets:
| CIDR Mask | Percentage | Cumulative |
|---|---|---|
| 32 | 4.9% | 4.9% |
| 31 | 8.3% | 13.2% |
| 30 | 3.2% | 16.4% |
| 29 | 4.2% | 20.6% |
| 28 | 1.3% | 21.9% |
| 27 | 2.2% | 24.1% |
| 26 | 0.9% | 25.0% |
| 25 | 0.8% | 25.8% |
| 24 | 0.9% | 26.7% |
| 23 | 5.7% | 32.3% |
| 22 | 5.7% | 38.0% |
| 21 | 1.4% | 39.4% |
| 20 | 4.2% | 43.6% |
| 19 | 1.0% | 44.6% |
| 18 | 1.1% | 45.7% |
| 17 | 0.4% | 46.1% |
| 16 | 0.1% | 46.2% |
Do delegations match authoritative NS records?
Here we examine whether delegation NS records (listed in the parent zone) match the authoritative NS records (found in the zone itself). We use the following six categories:
- Equal
The delegation records exactly match the authoritative records
- Disjoint
The delegation and authoritative records have no members in common
- Subset
The delegation records are a subset of the authoritative records
- Superset
The delegation records are a superset of the authoritative records
- Intersect
The delegation the authoritative records each contain members not found in the other set
- No Authoritative Nameservers
The supposedly-authoritative nameservers (learned from the parent zone) did not return any NS records for the zone
Note that we are only comparing nameserver names, rather than IP addresses. This may explain why there is a non-zero number of zones with Disjoint delegation/authoritative nameservers. The following table shows the percentage of each category:
| Equal | 979424 | 52.7% |
| No Authoritative Nameservers | 641448 | 34.5% |
| Subset | 102248 | 5.5% |
| Disjoint | 98615 | 5.3% |
| Superset | 21922 | 1.2% |
| Intersect | 13736 | 0.7% |
Do all nameservers return the same TTL for NS records?
Here we consider two slightly different cases. The first table shows the percentage of zones where the authoritative nameservers return NS records with mis-matched TTLs. This includes all NS records from all nameservers for a zone. The first column is the number of unique TTLs returned:
| 1 | 1205171 | 99.1% |
| 2 | 10672 | 0.9% |
| 3 | 95 | 0.0% |
| 4 | 6 | 0.0% |
Similarly, the second table shows the percentage of nameservers that return differing NS record TTLs in response to a single query:
| 1 | 2647240 | 99.8% |
| 2 | 6087 | 0.2% |
| 3 | 13 | 0.0% |
Are SOA values within their suggested ranges?
Refresh
RFC 1912 suggests that the SOA Refresh value should be at least 20 minutes and no longer than 12 hours. The following table shows the distribution of Refresh values:
| <20m | 55982 | 3.5% |
| 20m to 12h | 1374151 | 86.8% |
| >12h | 152388 | 9.6% |
Retry
RFC 1912 suggests that the SOA Retry value is typically less than the Refresh value. The following table shows the distribution of Retry values:
| retry <= refresh | 1833247 | 98.7% |
| retry > refresh | 24146 | 1.3% |
Expire
RFC 1912 suggests that the SOA Expire value should be at least 2 weeks and no longer than 4 weeks. The following table shows the distribution of Expire values:
| <2w | 1182384 | 74.7% |
| 2w to 4w | 151194 | 9.6% |
| >4w | 248943 | 15.7% |
Minimum
RFC 2308 suggests that the SOA Minimum value (aka the negative caching TTL) should be at least 1 hour and no longer than 3 hours. Common nameserver implementations usually enforce their own negative caching limits of 3 hours or less. The following table shows the distribution of Minimum values:
| <1h | 321872 | 20.3% |
| 1h to 3h | 355249 | 22.4% |
| >3h | 905400 | 57.2% |
Do serial numbers for a zone match?
| SAME | 1499140 | 94.7% |
| DIFF | 78533 | 5.0% |
| OFFBYONE | 4848 | 0.3% |
Note that this only includes zones that responded with at least one valid SOA record.
- SAME
all authoritative nameservers return the same serial number for the zone
- OFFBYONE
the minimum and maximum serial numbers among the set of authoritative nameservers differed by only one
- DIFF
one or more of the authoritative nameservers has a serial number different than the rest, where the difference is greater than one
How many zones have a lame server?
We test for lameness by sending an SOA query for a zone to an authoritative nameserver. If the response's RCODE is anything other than zero, we say the nameserver is lame. Here are the results:
| NOTLAME | 1343768 | 72.6% |
| LAME | 507796 | 27.4% |
Note this only includes nameservers for which we have an IP address. If a zone has 3 nameservers and we have can't get an IP address for 1 nameserver, the zone is included in the analysis with the 2 good nameservers. In other words, an unresolvable nameserver is not counted as a lame server.
Notes
Nameservers Density
We wish to note that some nameservers are authoritative for many different zones. This may affect certain measurements, such as the topological dispersion. For example, if a company with many zones happens to have all its nameservers on the same subnet, that may artificially inflate the percentages in the "Are nameservers topologically dispersed?" section.
To get an idea of the how many zones each nameserver hosts, note that the mean number of zones per nameserver is 37.2, while the median is 3.0. The following table shows how many zones each of the top 25 nameservers are authoritative for:
| NS Rank | Zone Count | CDF |
|---|---|---|
| 1 | 260956 | 3.75% |
| 2 | 260941 | 7.50% |
| 3 | 258776 | 11.22% |
| 4 | 258029 | 14.92% |
| 5 | 257099 | 18.62% |
| 6 | 184264 | 21.26% |
| 7 | 179092 | 23.84% |
| 8 | 48526 | 24.53% |
| 9 | 48252 | 25.23% |
| 10 | 48172 | 25.92% |
| 11 | 47062 | 26.59% |
| 12 | 44516 | 27.23% |
| 13 | 44514 | 27.87% |
| 14 | 41768 | 28.47% |
| 15 | 41598 | 29.07% |
| 16 | 36918 | 29.60% |
| 17 | 36834 | 30.13% |
| 18 | 36806 | 30.66% |
| 19 | 36796 | 31.19% |
| 20 | 36771 | 31.72% |
| 21 | 36621 | 32.24% |
| 22 | 36561 | 32.77% |
| 23 | 29796 | 33.20% |
| 24 | 29584 | 33.62% |
| 25 | 29263 | 34.04% |
© 2020 The Measurement Factory.