2015-02-25 Duane Wessels User reported a concern with the way dsc-xml-extractor.pl called the operating system 'mv' command. It has been replaced with Perl's File::Copy::mv(). 2013-03-19 Duane Wessels Fixed a bug in TCP reassembly when the DNS length field was split between two segments. 2012-08-27 Sebastian Castro Added 'tc_bit' indexer and dataset to track the frequency of responses having the TC bit set. Useful with DNSSEC signed zones. 2012-02-29 Duane Wessels Added 'qr_aa_bits' dataset and graph. It shows the distribution of QR/AA values in received messages and may be helpful in detecting reflector attacks targeting your name server. 2010-11-29 Duane Wessels The dnssec_qtype and dns_ip_version plots were not working due the presence of 'dataset' attributes in their definitions. Also fixed the chaos plot. 2010-11-04 Duane Wessels Added NSEC3 record type to the extractor. 2010-09-14 Duane Wessels A fix to put pcap interfaces into non-blocking mode, which is important when reading from multiple interfaces. 2010-08-13 Alexander Mayrhofer Added "country_index" to the collector which is an index based on country as returned by the GeoIP library. configure attempts to learn if you have GeoIP installed. 2010-08-13 Peter Koch Some minor fixes to the collector that allow it to read from pcap files on disk, rather than live packets from an interface. 2010-06-02 Henrik Kramsh?j Improvements for OpenBSD compatibility 2010-05-01 Duane Wessels Added NSEC3 to DNSSEC query types 2010-02-22 Duane Wessels Collector bug fixed: USE_IPV6=1 was not passed to .c files, which resulted in different-sized data structurs and crashes on CentOS. 2009-12-10 Duane Wessels Added "priming_queries" and "priming_responses" datasets in preparation for root zone signing. 2009-11-11 Duane Wessels On the presenter some of the "accum" graphs stopped working due to recent rewrites. These should now be working again. 2009-10-12 Duane Wessels The presenter debugging is now configurable via dsc-grapher.cfg. 2009-08-14 Duane Wessels In the presenter, there have been some significant changes to the perl modules so that they can be used in command line (ie non-CGI) utilities, both for creating graphs and for reading data. Most of these changes are purely internal. However, you may need to update your dsc-grapher.pl CGI program to be like presenter/grapher/dsc-grapher.pl.sample in the source distribution. 2009-04-15 Duane Wessels In the presenter's refile-and-grok.sh script, note that in some environtments it is useful to skip NODEs that don't have an incoming directory because they might have been "grokked" elsewhere and then rsync'd to you. 2009-02-27 Duane Wessels In the collector it is now necessary to include and . Not sure why it wasn't necessary before. Perhaps due to 2009-01-26 Hapy library upgrade. 2009-02-19 Duane Wessels In the collector, fixed some 64-bit free disk space calculations with casting. 2009-01-26 Duane Wessels Upgraded to more recent version of Hapy parsing library. 2008-12-31 Duane Wessels In the collector, changed the value of pcap_open_live() because we always try to read from all interfaces after select() returns. If we have multiple interfaces and one of them is quiet, and the timeout is too large, we'll drop packets on the other interface while waiting for the first to time out. Also added a pseudo-dataset that reports on pcap packets captured and dropped. 2008-12-02 Duane Wessels Grapher now preserves the order of the server list given in dsc-grapher.cfg. 2008-11-22 Duane Wessels Added some collector indexers and datasets for 2nd and 3rd level domains. 2008-10-02 Duane Wessels Made the 'no extractor for $dataset' error message non-fatal. Now, the unknown dataset will just be skipped and the remaining ones will be processed. 2008-09-30 Duane Wessels In the presenter, account for the possibility that the collector's clock is not in sync. 2008-09-18 Elmar Knipp Found bug in upload script where a $PROG.out prevents removal of a $YYYYMMDD directory. 2008-09-16 Duane Wessels New presenter feature: Node Merging. Now the DSC grapher can automatically "merge" multiple nodes so that they appear as one. For example instead of this dsc-grapher.cfg line: server S N1a N1b you can merge nodes "a" and "b" with this line: server S N1=N1a,N1b This feature is also useful for just renaming a node if you want it to be displayed as a different name than the directory where its files are. 2008-09-15 Duane Wessels On the collector, changed the sample config and upload scripts to use /usr/local/dsc/run as the default run_dir (was formerly /usr/local/dsc/var/run/). 2008-08-21 Duane Wessels Added an EDNS bufsiz indexer so we can collect buffer sizes advertised by clients. 2008-08-12 Duane Wessels Attempt to improve portability (on Solaris) by checking for libresolv, libnsl, and libsocket. Also check for statvfs() vs statvs() in configure. 2008-07-29 Duane Wessels Updates to collector/cron/upload-rsync.sh so that it behaves better with new date-based directories on both collector and presenter. 2008-06-27 Duane Wessels Added a source port range dataset (for about-to-be-announced vulnerability that can be mititgated by improving source port randomness). 2008-04-22 Duane Wessels Added ./configure script for presenter/dsc code. 2008-04-22 Duane Wessels Added support for capturing NCAP data streams. 2008-01-09 Duane Wessels Changed the location of the presenter's ".dat" files. Previously there were stored in a directory such as 20080109/qtype/qtype.dat and now they will be moved one directory up and stored as 20080109/qtype.dat. The source distribution includes a script found at presenter/grapher/update-dat-file-locations.sh that will traverse the /usr/local/dsc/data directory and move all the .dat files one level up. When upgrading to this version of DSC you should first stop the refile-and-grok.sh cron job, install the DSC software, and then run the update-dat-file-locations.sh script (after reading it). 2008-01-09 Duane Wessels Data passed from collector to presenter is now contained in a single XML file, rather than one XML file per dataset. This should significantly reduce filesystem pressure (i.e., 20x fewer files to be stored and sent) on both sides. 2008-01-07 Duane Wessels The collector now stores to-be-uploaded XML files in date-named subdirectories. This reduces filesystem pressure in the event that the collector cannot communicate with the presenter for an extended period of time. To accomodate this change, the upload-prep.sh script has been replaced with a Perl version, named upload-prep.pl. After installing DSC you should change your cron job to use the new script. 2008-01-04 Ken Keys Previously, a child was forked for each collection interval. Memory allocated for collected data was freed by simply exiting the child. But this left no way to preserve any data across intervals. Now, interval collectors do not fork. To avoid the tedious, slow, and error-prone nature of normal malloc-and-free memory management, we instead store non-persistent data in memory allocated from an "arena", which can be freed quickly and safely. We don't need to free individual allocations, but we do need to reset some pointers to allocated memory. TCP resets are now properly handled and TCP state is freed if a connection has been idle for too long. 2008-01-04 Duane Wessels The collector configuration (dsc.conf) has a new 'minfree_bytes' directive. If the amount of free disk space on dsc's partition falls below this limit, dsc will not write any XML files -- that data will be lost. The default value is 5 MB. 2007-12-14 Alexander Gall Added a 'dns_ip_version' indexer so that we can track DNS messages sent over v4 and v6. 2007-12-04 Duane Wessels Fixed a collector bug when listening on multiple interfaces of different datalink types (e.g., eth0 and lo). 2007-11-26 Duane Wessels Added a 'hide_nodes' feature to dsc-grapher.cfg. If 'hide_nodes' is given, then the navigation menu will not display the nodes underneath a server. Note, however, that knowledgeable users could still view individual node data by manually if they know the names by specifying it in the URL query terms. 2007-10-09 Duane Wessels Changed the presenter to not use "data URIs" by default. Now Internet Explorer users should be happier. 2007/06/14 Duane Wessels Added -f command line option to keep 'dsc' collector as a foreground process. 2007-06-11 Duane Wessels dsc-xml-extractor.pl now looks for incoming XML files in "incoming/YYYY-MM-DD" directories. 2007-06-07 Duane Wessels New TCP code did not compile cleanly on Linux. Added some #ifdefs and macros for Linux compatibility on TCP headers. 2007-05-15 Ken Keys, Duane Wessels TCP support New 'transport' indexer for DNS messages. Indicates whether message arrived over UDP or TCP. 2007-04-04 Duane Wessels IPv6 support New 'ip_version' indexer. Indicates whether DNS message arrived via IPv4 or IPv6.