Skip site navigation (1)Skip section navigation (2)

FreeBSD Manual Pages


home | help
DNSCheck::Overview(3) User Contributed Perl DocumentationDNSCheck::Overview(3)

       The DNSCheck System

       The central part	of the DNSCheck	system are the Perl modules that
       perform the actual tests. All of	them should be used by creating	a object and letting it create	other objects as needed	and/or
       requested. These	modules	will make lots of queries to DNS servers all
       around the world, and sooner or later return a result in	the form of a
       list of messages.

       At this level, the libs can be used for any number of tasks. In the
       examples	directory in the source	distribution there is, for example, a
       script that lets	one monitor the	status of a domain via the Nagios
       monitoring system.

       The system as distributed, however, contains more than the lib. There
       are also	a couple of programs and a web interface. We will not talk
       about the web interface here, except to mention that it like the	rest
       of the programs communicate with	the other parts	via the	database
       tables to be mentioned later.

   The programs
       The dispatcher

       The program that	actually runs tests is dnscheck-dispatch. It polls the
       table "queue" table in the database every 0.25 seconds, and forks
       children	to run tests on	what it	finds there. It	monitors the state of
       the children, and cleans	up after they exit. If a child exits
       uncleanly, the test it was running is re-scheduled. If the same domain
       causes unclean exits five times or more,	that domain will be
       blacklisted and will not	be tested again	until the dispatcher is

       If the dispatcher receives a TERM signal, it will stop starting new
       tests, wait for the ones	currently running to exit and then exit	in
       good order. This	may at worst take several minutes.

       If the dispatcher receives a HUP	signal,	it will	do an orderly exit as
       above, and then re-execute itself. If it	was not	originally started
       with a full pathname, the re-execution may fail.

       The dispatcher is written to do a small number of tasks (get queue
       entry, start child, clean up after it) in as robust fashion as
       possible. As of this writing, the program appears to be very reliable
       as long as the database server is not too heavily loaded. Exactly what
       "too heavily loaded" means in practice will vary, of course, but	as a
       rule of thumb, if a simple UPDATE takes more than a second to finish
       the system is almost certainly there.

       It is vital that	the queue table	use the	InnoDB storage engine. If it
       is MyISAM, there	will be	data corruption	under heavy load. Other
       storage engines with sane transaction handling should also work,	but
       have not	been tested.

       At the moment, only MySQL is supported. Patches to add support for more
       databases are welcome.

       The dispatcher's	child processes	run the	default	tests on the given
       domain, write the results to the	"tests"	and "results" tables in	the
       database	and then exits.

       The zone	change detector

       One of the programs that	insert entries into the	"queue"	table is
       dnscheck-zonediff. This program does a zone transfer of a specified
       domain and looks	at all the "NS"	and "DS" records. For ever subdomain,
       it'll concatenate the textual representations of	the records and
       calculate an MD5	sum of the resulting string. If	that sum is not
       identical with what it was the previous time the	program	was run, the
       subdomain in question will be scheduled for testing (that is, it'll be
       written to the "queue" table, where the dispatcher will pick it up).

       The change detector stores data between runs in a simple	text file,
       with a zone name	and its	corresponding MD5 sum on each line.

       At the moment, the time it takes	to do the AXFR is limited by the speed
       at which	the Perl module	Net::DNS can create new	records. It also uses
       a fair bit of RAM (100-150MB for	around 750 thousand domains), so if
       you're planning to run it against ".com"	you'll want a pretty hefty

       The results mailer

       The dnscheck-12hourmailer script	reads results of tests from the
       database	tables "tests" and "results", aggregates them per registrar
       and sends an email to each registrar with a summary of serious problems
       found among their domains.

       This is pretty specific to ".SE", particularly the bit that maps	domain
       names to	registrars and fetches contact information for the registrars.
       If you want to do something similar to this, you'll probably need to
       modify the code of the script. At the moment, the emails	are also
       generated entirely programmatically, so changing	their formatting also
       involves	changing code.

       The preflight script

       dnscheck-preflight is used by the web GUI. It takes a name as a command
       line argument and prints	the string TRUE	if it's	a domain and FALSE

       The hostsyntax script

       dnscheck-hostsyntax is also used	by the web GUI.	It takes a name	as a
       command line argument, and prints the string TRUE if it is a
       syntactically valid hostname and	FALSE otherwise.

       The CLI interface

       dnscheck	is a command line tool to check	domains. Run it	with the
       switch "--help" to get a	short description of how to use	it.

   The database
       It's entirely possible to use the lib part of DNSCheck without a
       database. All the surrounding functionality, however, needs one.	All
       communication between various parts of the entire system	is done
       entirely	through	database tables.  While	this may not be	the most
       convenient interface ever, it does mean that anything that can talk to
       MySQL can be a part of the system, which	is useful.

       There are a number of tables. Let's start looking at them from the top
       of, so to speak,	the food chain.

       The source table

       Anythingt that can write	to the "queue" table can be a source of
       domains to be tested. In	order to make it possible to keep track	of
       which source requested which test, there	is the "source"	table. It has
       only three fields, and is supposed to be	mostly read-only. The fields
       are "id", "name"	and "contact". The id is an autoincrementing integer
       as usual. The name is a human-readable name for the source. The contact
       is some sort of human-useable contact information for the source	in
       question	(email address,	phone number, ham radio	call sign,...).

       One example of the use of source	information is the mailer described
       above, which only mails results about tests that	were scheduled for
       testing by the zone change detector, not	once that came from (for
       example)	the web	GUI.

       The queue table

       This table has six fields apart from the	autoincrementing "id". There
       is "domain", which is the name of a domain to be	tested.	There is
       "priority", which is an integer.	dnscheck-dispatch will always pick the
       queue entry with	the highest priority first (and	if there are several
       with the	same priority, the one with the	lowest id). There is
       "inprogress", which is NULL to begin with and gets a timestamp written
       to it by	the dispatcher when it's been scheduled	for testing. There is
       a "tester_pid" field, where the particular child	process	that's running
       the test	for the	domain writes it's system PID (if we ever need to have
       more than one machince running tests, thre maybe	a host field added
       here). "source_id" pints	into the source	table.	"source_data" is any
       data specific to	this test that is relevant for the specific source.
       The two source fields are simply	copied over to the "tests" table when
       a test is started.

       The tests table

       This table has one row for every	test run, an contains information on
       the domain tested, the start and	finish times of	the test, a summary of
       the results and the source information that was copied from the queue
       table. The fields are: "id" as usual, "domain" for the domain name,
       "begin" for the start time (with	whole-second resolution), "end"	for
       the finish time,	"count_critical", "count_error", "count_warning",
       "count_notice" and "count_info" for the number of problems at the
       respecive severity levels, "source_id" to point into the	source table
       and finally "source_data" to hold whatever data that source wants.

       The results table

       This table has a	number of rows for every test. They reflect the	list
       of results returned after a test	is run,	and several of the fields are
       rather specific to the web GUI application. The fields that can be
       useful to others	are: "test_id",	which points into the "tests" table.
       "timestamp" with	is the time at which the log entry was entered (with
       microsecond resolution).	 "level" which is one of the all-caps strings
       CRITICAL, ERROR,	WARNING, NOTICE	or INFO. "message" which is the
       message tag that	describes the reason the log entry was made. "arg0" to
       "arg9", which are arguments to the message.

       The message tags	are specified in YAML files.

       The messages table

       The messages table contains a copy of a localy YAML file. It is used by
       the web GUI.

       The delegation_history table

       This table contains information about which nameservers has
       historically been authoritative for domains. Some tests use it, if it's

       The nameservers and nsp tables

       The function of these tables have been obsoleted	by the registrar

       The domains table

       A list of domains. More of a convenience	thing than an actual part of
       the DNSCheck system.

       Needed permissions

       The various programs need different sets	of permissions in the database
       in order	to work	correctly. It breaks down as follows:

       o   Everything that schedules tests needs INSERT, UPDATE	and SELECT on
	   "sources" and INSERT	on "queue".

       o   Everything that reads test results needs SELECT on "sources",
	   "tests" and "results".

       o   Anything that runs tests needs SELECT on "delegation_history" (or
	   no database access at all).

       o   dnscheck-dispatcher needs a lot: INSERT, SELECT, UPDATE and DELETE
	   on "queue", INSERT, SELECT and UPDATE on "tests", INSERT on
	   "results" plus everything listed above.

       In practice, the	easiest	reasonably secure setup	is to give the web GUI
       one user	with very limited permissions (INSERT, UPDATE, SELECT on
       "sources" and INSERT on "queue"), and one user with full	rights on all
       tables for the rest of the system. Indeed, since	all of the programs
       use the database	connection info	given in the site_config.yaml file,
       you will	need to	point out a different such file	to any programs	you
       want to use different connection	information.

       The system uses a number	of files.

       config.yaml and site_config.yaml

       These contain configuration information in YAML format. The
       "config.yaml" file is installed with DNSCheck, and should not be
       modified	(if you	do modify it, your changes will	be overwritten when
       you upgrade DNSCheck).  "site_config.yaml" overrides what is in
       "config.yaml", and is where local changes (such as database access
       information and TSIG keys) should be entered.  It will not be touched
       by the installation process.

       One key that is usually not in either of	these files but	that can be
       added if	needed is the "disable"	key. It	has two	levels under it, being
       the name	a test module (converted to lowercase) and the name of a
       method in that module. If the value for the method is true, that	method
       will not	be run.	Note that no kind of sanity checking is	done, and that
       some tests rely on each other, so disabling one may make	others give
       false results. Be cautious and know what	you're doing. As an example,
       the following would disable all DNSSEC tests:

		test: 1

       All test	methods	mentioned in the documentation for the respective
       modules are possible to disable this way.

       policy.yaml and site_policy.yaml

       "policy.yaml" specifies the severity of various problems. As
       distributed, it represents .SE's	view on	best practices for DNS.	If you
       want to change anything,	do so in "site_policy.yaml", or	your changes
       will be overwritten on the next install.

       locale/en.yaml and locale/sv.yaml

       These files contain information on how to transform the raw test	log
       entries into human-readable messages. If	you want to do your own
       translation, you	need to	know that the "format" key is a	sprintf()
       format string, and "args" the number of arguments after the format. If
       the latter number does not match	the Perl code, a warning will be
       printed at runtime.

perl v5.32.1			  2010-06-29		 DNSCheck::Overview(3)


Want to link to this manual page? Use this URL:

home | help