researchtoolssurveys DNSFactory

DNS SURVEY: JUNE 2005

Goals

This study, commissioned by Infoblox, is designed to answer the following questions:

  • How many nameservers advertise their software version?
  • How many nameservers allow recursion?
  • How many nameservers allow a zone transfer?
  • Are nameservers topologically dispersed?
  • Do delegations match authoritative NS records?
  • Do all nameservers return the same TTL for NS records?
  • Are SOA values within their suggested ranges?
  • Do serial numbers for a zone match?
  • How many zones have a lame server?

Methodology

Zones To Check

Our goal is to check more than 1,000,000 DNS zones. Since large lists of hostnames are a little hard to come by, we used a copy of the COM, NET, and ORG zones from August 2002.

We concatenated these three zones and then randomized their order. We counted 27,266,895 total registered domains in these three zones.

From the randomized list we selected a random sample of 10%. We then queried a local resolver for each zone to make sure that it still exists in the global DNS. This left us with 1,640,180 valid zones. Note that with this technique we have only second-level domain names.

Due to a problem that we did not detect until the scan was over, our script tested only names in the COM zone. This still leaves us with 1,311,909 names.

Types of Queries

For each zone we:

  1. Get the list of delegated nameservers from the parent zone.
  2. Get the list of authoritative nameservers, starting with the delegations from step 1.
  3. Test each authoritative nameserver for lameness.
  4. Test each authoritative nameserver for recursion.
  5. Test each authoritative nameserver for TCP support.
  6. Test each authoritative nameserver for zone transfer.
  7. Send a version.bind query to each authoritative nameserver.

Queries are generated with the Net::DNS Perl module. This means, for example, that unanswered queries are retransmitted according to the default timeouts.

We test for lameness by sending an SOA query for a zone to an authoritative nameserver. If the nameserver does not return any answer RRs, we sleep for 15 seconds and try again. We say the nameserver is lame if we do not get any answer RRs after three attempts.

We test recursion by sending three queries to each authoritative nameserver. These are A queries for www.yahoo.com, and www.microsoft.com. We use popular names to increase the chance of getting an answer. That is, we believe that the nameservers for the popular names have high-availability and throughout the Internet. We say the nameserver allows recursion if it provides at least one A RR for each query. This means, for example, if a nameserver only returned 2 out of 3 queries, it would not get marked as recursive. This was done to account for nameservers that, for whatever reason, think they are authoritative for one of the query names.

We test for TCP support by setting the `usevc' option to Net::DNS::Resolver and then making an SOA query for the zone. We say the nameserver supports TCP if it returns at least one answer RR. Each nameserver is tested only once.

We test for zone transfer with Net::DNS::Resolver's axfr_start and axfr_next functions. We say the nameserver allows the zone transfer if it returns at least one RR. Each nameserver is tested only once, even if it is authoritative for many zones.

We send version.bind queries to each authoritative nameserver just once.

Query Sources

We distributed our queries from approximately 250 IP addresses in the same /24 subnet.

Results

The scan ran from 2005-06-27 12:42 UTC to 2005-06-29 12:19 UTC.

We analyzed 1,311,909 zones and 260,098 nameservers.

How many nameservers advertise their software version?

About 36% of nameservers do not openly advertise their software version. Most of those that do are running BIND software:

NORMAL_BIND13336351.3%
NOANSWER6064523.3%
NOREPLY3405413.1%
OTHER207718.0%
NOADDRESS59102.3%
EMPTY28891.1%
SEMI_NORMAL_BIND23170.9%
NSD650.0%
TINYDNS420.0%
ULTRADNS410.0%
NORMAL_BIND

the version string matches the pattern /^[489]\.[0-9]/

SEMI_NORMAL_BIND

the version string matches the pattern /^{named,bind} [489]\.[0-9]/

NOREPLY

we did not receive a DNS reply to the query

NOANSWER

we received a reply, but it did not contain any answer RRs

EMPTY

the version.bind reply was the empty string

NOADDRESS

we could not determine the IP address for the nameserver

How many nameservers allow recursion?

The following table shows how many nameservers appear to recursively resolve domain names on our behalf:

YES19151075.3%
NO6267724.7%
NOADDRESS5910

NOADDRESS means we could not determine the IP address for the nameserver.

We also have data on how many DNS replies came back with the RA bit set:

Query NameSetClear
www.microsoft.com84.0%16.0%
www.google.com83.7%16.3%
www.yahoo.com83.7%16.3%

Note that the percentage in the first table is lower. This may be due to our requirement to receive answers to all three names. It may also be due to nameservers that set the RA bit, but don't actually provide recursion for some reason.

How many nameservers allow a zone transfer?

The following table shows how many nameservers allowed us to transfer one of their zones:

YES10527041.4%
NO14891758.6%
NOADDRESS5910

NOADDRESS means we could not determine the IP address for the nameserver.

Related to this measurement is the number of nameservers that allow TCP transactions at all:

YES19930678.4%
NO5488121.6%
NOADDRESS5910

Are nameservers topologically dispersed?

RFC 2182 states: secondary servers must be placed at both topologically and geographically dispersed locations on the Internet, to minimize the likelihood of a single failure disabling all of them. While we do not test for geographic diversity, we can easily examine the topologic aspects of a zone's nameservers. Our assumption that topologic locality implies geographic locality is is true in most, but not quite all cases. The following table shows the percentage of zones having all their authoritative nameservers within the given CIDR-sized subnets:

CIDR MaskPercentageCumulative
323.9%3.9%
3110.5%14.4%
305.2%19.6%
294.8%24.4%
282.1%26.4%
272.4%28.8%
261.6%30.4%
251.0%31.4%
241.4%32.8%
235.3%38.1%
222.5%40.7%
216.0%46.7%
206.2%52.9%
191.6%54.5%
182.0%56.5%
170.5%56.9%
160.3%57.3%

Do delegations match authoritative NS records?

Here we examine whether delegation NS records (listed in the parent zone) match the authoritative NS records (found in the zone itself). We use the following six categories:

Equal

The delegation records exactly match the authoritative records

Disjoint

The delegation and authoritative records have no members in common

Subset

The delegation records are a subset of the authoritative records

Superset

The delegation records are a superset of the authoritative records

Intersect

The delegation the authoritative records each contain members not found in the other set

No Authoritative Nameservers

The supposedly-authoritative nameservers (learned from the parent zone) did not return any NS records for the zone

Note that we are only comparing nameserver names, rather than IP addresses. This may explain why there is a non-zero number of zones with Disjoint delegation/authoritative nameservers. The following table shows the percentage of each category:

Equal78473560.5%
No Authoritative Nameservers29248022.5%
Subset833316.4%
Disjoint821716.3%
Superset311742.4%
Intersect241401.9%

Do all nameservers return the same TTL for NS records?

Here we consider two slightly different cases. The first table shows the percentage of zones where the authoritative nameservers return NS records with mis-matched TTLs. This includes all NS records from all nameservers for a zone. The first column is the number of unique TTLs returned:

199213798.7%
2128861.3%
35000.0%
4220.0%
540.0%
710.0%

Similarly, the second table shows the percentage of nameservers that return differing NS record TTLs in response to a single query:

1253738599.9%
222760.1%
3160.0%
770.0%

Are SOA values within their suggested ranges?

Refresh

RFC 1912 suggests that the SOA Refresh value should be at least 20 minutes and no longer than 12 hours. The following table shows the distribution of Refresh values:

<20m515964.3%
20m to 12h102912585.3%
>12h12613210.5%

Retry

RFC 1912 suggests that the SOA Retry value is typically less than the Refresh value. The following table shows the distribution of Retry values:

retry <= refresh117945697.7%
retry > refresh273972.3%

Expire

RFC 1912 suggests that the SOA Expire value should be at least 2 weeks and no longer than 4 weeks. The following table shows the distribution of Expire values:

<2w89963674.5%
2w to 4w1021808.5%
>4w20503717.0%

Minimum

RFC 2308 suggests that the SOA Minimum value (aka the negative caching TTL) should be at least 1 hour and no longer than 3 hours. Common nameserver implementations usually enforce their own negative caching limits of 3 hours or less. The following table shows the distribution of Minimum values:

<1h24174820.0%
1h to 3h33873828.1%
>3h62636751.9%

Do serial numbers for a zone match?

SAME103681179.9%
NOADDRESS22842717.6%
DIFF269212.1%
OFFBYONE58720.5%
SAME

all authoritative nameservers return the same serial number for the zone

NOADDRESS

we could not determine the IP address for one or more of the authoritative nameservers

OFFBYONE

the minimum and maximum serial numbers among the set of authoritative nameservers differed by only one

DIFF

one or more of the authoritative nameservers has a serial number different than the rest, where the difference is greater than one

How many zones have a lame server?

NOTLAME108672683.7%
NOERROR1240169.6%
NOREPLY761545.9%
SERVFAIL33310.3%
REFUSED16610.1%
NXDOMAIN7410.1%
NOTIMP20.0%
MULTIPLE52260.4%

For zones with one lame server, we list the RCODE from the lame reply. Zones with more than one lame server are represented by the MULTIPLE line.

Note this only includes nameservers for which we have an IP address. If a zone has 3 nameservers and we have can't get an IP address for 1 nameserver, the zone is included in the analysis with the 2 good nameservers. In other words, an unresolvable nameserver is not counted as a lame server.

Notes

Nameservers Density

We wish to note that some nameservers are authoritative for many different zones. This may affect certain measurements, such as the topological dispersion. For example, if a company with many zones happens to have all its nameservers on the same subnet, that may artifically inflate the percentages in the "Are nameservers topologically dispersed?" section.

To get an idea of the how many zones each nameserver hosts, note that the mean number of zones per nameserver is 37.2, while the median is 3.0. The following table shows how many zones each of the top 25 nameservers are authoritative for:

NS RankCountCDF
12537103.57%
22537057.13%
325369310.70%
425363514.26%
525359017.83%
66873618.79%
76873219.76%
86794820.72%
96794821.67%
104930622.36%
114930023.06%
124760823.73%
134739624.39%
144725225.06%
154704025.72%
163839026.26%
173612526.77%
183612527.27%
193611527.78%
203611528.29%
213608028.80%
222713029.18%
232710629.56%
242698229.94%
252662230.31%

Announcement to NANOG

We announced this survey to the NANOG mailing list on June 24 (3 days beforehand) including the addresses where queries would come from. It is possible that some people may have configured filters on their networks to block our probes. However, we do not expect that those doing so would have much effect on the results of this survey.

Complaints Received

As a result of this survey we received four abuse complaints via email (3 from humans, 1 automated). All complaints were about zone transfer attempts. After explaining the survey, the humans were very understanding.

Prior to the survey we added comments to the ARIN whois database for our source netblock. The comments provided a brief explanation of the survey and included our contact details. Without these comments and descriptive web page, we probably would have received more complaints.

© 2020 The Measurement Factory.