dns_sprockets is a command-line framework for loading and validating DNS zones. It is written in Python and uses the excellent dnspython library for much of its functionality.
"Why call it dns_sprockets?" Well, if you think of a DNS zone as being a chain of resource records (thinking especially of NSEC3 records), this application is a set of sprockets that tests every link of the chain. Hey, it kind of works ;)
Possible users include DNS code developers and quality assurance, internet customer service, system administrators, and end-users who are interested to know if their DNS zones are valid.
The command-line tool returns useful return codes, supporting automated build systems. It is built around the concept of plug-ins for implementing both the zone loading and zone validating functions, and easily allows end-users the ability to define new loaders and validators.
Zones may be loaded using various means. The framework supports "loader" plug-ins that pull DNS zone data from any source. Initially provided are 'File' and 'Xfr' plugins to pull zone data from host files and XFR servers, respectively.
Validations are modular pieces of code that may be selectively enabled. The framework supports "validator" plug-ins that operate in one of four views:
- Zone: Some aspect of the entire zone is validated.
- Node: Every node (i.e. list of RRSets with the same owner name) can be validated.
- RRSet: Every RRSet can be validated.
- Record: Every DNS record can be validated.
Initially provided in this package are some basic zone validators, and a fairly complete set of DNSSEC-type zone validators.
The Node, RRSet, and Record views may be optionally filtered by one or more resource record types, to simplify and focus the validation code. Additionally, each validator can be marked to run for non-DNSSEC zones, NSEC1-style DNSSEC zones, or NSEC3-style DNSSEC zones.
Easy! Use pip (preferably in a virtual python environment):
$ pip install dns_sprockets
Once installed, simply issue the dns_sprockets command. For example to get a usage message:
$ dns_sprockets --help
# dns_sprockets (1.1.8) - A DNS Zone validation tool
usage: dns_sprockets.py [-h] [-z s] [-l s] [-s s] [-i s] [-x s] [-d s] [-f s]
[-e] [-v]
optional arguments:
-h, --help show this help message and exit
-z s, --zone s Name of zone to validate [www.ultradns.com]
-l s, --loader s Zone loader method to use (one of: file, xfr) [xfr]
-s s, --source s Loader source to use [127.0.0.1#53]
-i s, --include s Only include this validator (can use multiple times)
-x s, --exclude s Exclude this validator (can use multiple times)
-d s, --define s Define other params (can use multiple times)
-f s, --force-dnssec-type s
Use DNSSEC type (one of: detect, unsigned, NSEC,
NSEC3) [detect]
-e, --errors-only Show validation errors only [False]
-v, --verbose Show detailed processing info [False]
Use @filename to read some/all arguments from a file.
Use -d's to define optional, module-specific parameters if desired (e.g. to tell
'xfr' loader to use a specific source address, use "-d xfr_source=1.2.3.4").
The optional parameters are listed under each loader and validator description
in DEFINE lines, if available. The "short form" of parameter names (i.e.
without the module name prefix) may be used instead, if multiple modules define
the same "short form" name (e.g. -d now=20160328120000 can be used, and will set
both 'rrsig_missing' and 'rrsig_orphan' validators' rrsig_missing_now and
rrsig_orphan_now values, respectively).
By default, all tests are run. Use -i's to explicitly specify desired tests,
or -x's to eliminate undesired tests.
The list of available loaders is:
---------------------------------------------------------------------------
LOADER: file - Loads a zone from a file in AXFR-type or Bind host-type format.
DEFINE: (file_)allow_include - Allow file to include other files (default=1)
DEFINE: (file_)rdclass - Class of records to pull (default=IN)
LOADER: xfr - Loads a zone by XFR from a name server.
DEFINE: (xfr_)af - The address family to use, AF_INET or AF_INET6 (default=None)
DEFINE: (xfr_)keyalgorithm - The TSIG algorithm to use, one of: HMAC-MD5.SIG-ALG.REG.INT. hmac-sha1. hmac-sha224. hmac-sha256. hmac-sha384. hmac-sha512. (default=HMAC-MD5.SIG-ALG.REG.INT.)
DEFINE: (xfr_)keyname - The name of the TSIG to use (default=None)
DEFINE: (xfr_)keyring - The TSIG keyring to use, a text dict of name->base64_secret e.g. "{'n1':'H477A900','n2':'K845CL21'}" (default=None)
DEFINE: (xfr_)lifetime - Total seconds to wait for complete transfer (default=None)
DEFINE: (xfr_)rdclass - Class of records to pull (default=IN)
DEFINE: (xfr_)rdtype - Type of XFR to perform, AXFR or IXFR (default=AXFR)
DEFINE: (xfr_)serial - SOA serial number to use as base for IXFR diff (default=0)
DEFINE: (xfr_)source - Source address for the transfer (default=None)
DEFINE: (xfr_)source_port - Source port for the transfer (default=0)
DEFINE: (xfr_)timeout - Seconds to wait for each response message (default=5.0)
DEFINE: (xfr_)use_udp - Use UDP for IXFRing (default=0)
The list of available validators is:
---------------------------------------------------------------------------
TEST: dnskey_bits (REC_TEST[DNSKEY]) - Checks DNSKEY flags and protocol.
TEST: dnskey_origin (ZONE_TEST) - Checks for a ZSK at zone origin.
TEST: dnssectype_ambiguous (ZONE_TEST) - Checks for existence of both NSEC and NSEC3 in the zone.
TEST: ns_origin (ZONE_TEST) - Checks for at least one NS at zone origin.
TEST: nsec3_chain (ZONE_TEST) - Checks for valid NSEC3 chain.
TEST: nsec3_missing (RRSET_TEST) - Checks that all (non-NSEC3/RRSIG, non-delegated) RRSets are covered with an NSEC3.
TEST: nsec3_orphan (REC_TEST[NSEC3]) - Checks for orphan or invalid-covers NSEC3s.
TEST: nsec3param_origin (ZONE_TEST) - Checks for an NSEC3PARAM at zone origin for nsec3-type zones.
TEST: nsec_chain (ZONE_TEST) - Checks for valid NSEC chain.
TEST: nsec_missing (RRSET_TEST) - Checks that all (non-NSEC/RRSIG, non-delegated) RRSets are covered with an NSEC.
TEST: nsec_orphan (REC_TEST[NSEC]) - Checks for orphan or invalid-covers NSECs.
TEST: nsecx_ttls_match (REC_TEST[NSEC,NSEC3]) - Checks that NSECx TTL's match SOA's minimum.
TEST: rrsig_covers (REC_TEST[RRSIG]) - Checks RRSIG's don't cover RRSIG's.
TEST: rrsig_missing (RRSET_TEST) - Checks that all (non-RRSIG, non-delegated) RRSets are covered with an RRSIG.
DEFINE: (rrsig_missing_)now - Time to use for validating RRSIG time windows, e.g. 20150101123000 (default=None)
DEFINE: (rrsig_missing_)now_offset - Number of seconds to offset the "now" value, e.g. -86400) (default=None)
TEST: rrsig_orphan (REC_TEST[RRSIG]) - Checks for orphan RRSIGs.
DEFINE: (rrsig_orphan_)now - Time to use for validating RRSIG time windows, e.g. 20150101123000 (default=None)
DEFINE: (rrsig_orphan_)now_offset - Number of seconds to offset the "now" value, e.g. -86400) (default=None)
TEST: rrsig_signer_match (REC_TEST[RRSIG]) - Checks RRSIG signers match the zone.
TEST: rrsig_time (REC_TEST[RRSIG]) - Checks RRSIG's inception <= expiration.
TEST: rrsig_ttls_match (REC_TEST[RRSIG]) - Checks RRSIG TTL's match original and covered TTL's.
TEST: soa_origin (ZONE_TEST) - Checks for an SOA at zone origin.
TEST: soa_unique (ZONE_TEST) - Checks for a single SOA in the zone.
Let's say you want to validate and only see errors an NSEC3-style DNSSEC zone called "example", from a file, and wish to run all available/applicable validations. Since this will check RRSIG signatures, you'll need to add a few defines to properly state the "now" time to use for two of the validators. Assuming a bash-like shell:
$ ZONE_FILE=$VIRTUAL_ENV/lib/python2.7/site-packages/dns_sprockets_lib/tests/data/rfc5155_example.
$ TIME_NOW=20100101000000
$ dns_sprockets -z example -l file -s $ZONE_FILE -e \
-d rrsig_missing_now=$TIME_NOW -d rrsig_orphan_now=$TIME_NOW
# dns_sprockets (1.0.0) - A DNS Zone validation tool
# Checking zone: example.
# Loader: file from: rfc5155_example. elapsed=0.018354 secs
# Zone appears to be DNSSEC type: NSEC3
# Extra defines: ['rrsig_missing_now=20100101000000', 'rrsig_orphan_now=20100101000000']
# Skipping test: nsec_chain (DNSSEC type for zone: NSEC3, for test: NSEC)
# Skipping test: nsec_missing (DNSSEC type for zone: NSEC3, for test: NSEC)
# Skipping test: nsec_orphan (DNSSEC type for zone: NSEC3, for test: NSEC)
# Running tests: ['dnskey_origin', 'dnssectype_ambiguous', 'ns_origin', 'nsec3_chain', 'nsec3param_origin', 'soa_origin', 'soa_unique', 'nsec3_missing', 'rrsig_missing', 'dnskey_bits', 'nsec3_orphan', 'nsecx_ttls_match', 'rrsig_covers', 'rrsig_orphan', 'rrsig_signer_match', 'rrsig_time', 'rrsig_ttls_match']
# END RESULT: 0 ERRORS in 229 tests
# TOTAL ELAPSED TIME: 0.063526 SECS LOAD TIME: 0.018354 SECS TEST TIME: 0.045172 SECS
$ echo $?
0
UPDATE New in version 1.1.8: Can now just specify "-d now=$TIME_NOW" as a shortcut for "-d rrsig_missing_now=$TIME_NOW -d rrsig_orphan_now=$TIME_NOW"
OK, all tests passed, but that's not too interesting. Let's repeat that, except with a slightly modified zone file: one of the NSEC3's (and its associated RRSIG record) has been removed:
$ ZONE_FILE=$VIRTUAL_ENV/lib/python2.7/site-packages/dns_sprockets_lib/tests/data/rfc5155_example._nsec3_missing
$ dns_sprockets -z example -l file -s $ZONE_FILE -e \
-d rrsig_missing_now=$TIME_NOW -d rrsig_orphan_now=$TIME_NOW
# dns_sprockets (1.0.0) - A DNS Zone validation tool
# Checking zone: example.
# Loader: file from: dns_sprockets_lib/tests/data/rfc5155_example._nsec3_missing elapsed=0.023993 secs
# Zone appears to be DNSSEC type: NSEC3
# Extra defines: ['rrsig_missing_now=20100101000000', 'rrsig_orphan_now=20100101000000']
# Skipping test: nsec_chain (DNSSEC type for zone: NSEC3, for test: NSEC)
# Skipping test: nsec_missing (DNSSEC type for zone: NSEC3, for test: NSEC)
# Skipping test: nsec_orphan (DNSSEC type for zone: NSEC3, for test: NSEC)
# Running tests: ['dnskey_origin', 'dnssectype_ambiguous', 'ns_origin', 'nsec3_chain', 'nsec3param_origin', 'soa_origin', 'soa_unique', 'nsec3_missing', 'rrsig_missing', 'dnskey_bits', 'nsec3_orphan', 'nsecx_ttls_match', 'rrsig_covers', 'rrsig_orphan', 'rrsig_signer_match', 'rrsig_time', 'rrsig_ttls_match']
TEST nsec3_chain(ZONE(example. IN)) => FAIL: Chain broken at R53BQ7CC2UVMUBFU5OCMM6PERS9TK9EN (next=T644EBQK9BIBCNA874GIVR6JOJ62MLHV doesn't exist)
TEST nsec3_missing(RRSET(xx.example. IN A)) => FAIL: No NSEC3's found for name: t644ebqk9bibcna874givr6joj62mlhv.example.
TEST nsec3_missing(RRSET(xx.example. IN HINFO)) => FAIL: No NSEC3's found for name: t644ebqk9bibcna874givr6joj62mlhv.example.
TEST nsec3_missing(RRSET(xx.example. IN AAAA)) => FAIL: No NSEC3's found for name: t644ebqk9bibcna874givr6joj62mlhv.example.
# END RESULT: 4 ERRORS in 221 tests
# TOTAL ELAPSED TIME: 0.064603 SECS LOAD TIME: 0.023993 SECS TEST TIME: 0.040610 SECS
$ echo $?
4
This time, we get errors from two validators. The nsec3_chain validator issues a "chain broken" error, and the nsec3_missing validator sees three RRSet's with the same owner name that are "not covered" by the missing NSEC3.
Incidentally, these two data files (and others) are included in the package for unit testing purposes, but can be useful to play with to see how dns_sprockets reports various problems.
The application returns a numerical value back to the user:
- 0 If there were no failed validations.
- 1-254 The number of failed validations, up to a limit of 254.
- 255 A special code for fatal exceptions.
The following is a non-exhaustive list of things to do (help anyone?):
- Respect the "opt-out" flag in NSEC3 records; right now, assuming none are opt-out.
- More loaders and (especially) validators!
- More real-world trials.
This long section discusses dns_sprockets for those who may be interested in adding more loaders or validators. If that's you, great! Please consider contributing your work to the project, it is most welcome! Especially welcome are unit tests that accompany any new code! (currently using Nose for testing).
Inspiration for this application comes from a similar tool written in Perl called donuts. It too uses the concept of plugins for its validators.
This framework essentially revolves around the two types of plugins: Loaders and validator plugins, which are stored in two project subfolders (dns_sprockets_lib/loaders and dns_sprockets_lib/validators, respectively). At runtime, the app scans both folders and makes their contents available for use:
A note on the naming conventions: plugins are stored in files with underscore-style names (e.g. nsec3_chain.py) and are expected to contain a class that implements the plugin, with a camelcase-style name that corresponds to the file name (e.g. Nsec3Chain).
The main logic of the app resides in the DNSSprocketsImpl.run() method (in dns_sprockets_lib/dns_sprockets_impl.py). Pseudo-code is:
- Scan zone loaders and load them into memory as Python classes.
- Create an instance of the specified zone loader.
- Scan validators and load them into memory as Python classes.
- Instantiate specified validators and categorize by validator type.
- Run the specified zone loader instance to obtain a dns.zone.Zone object.
- Construct a "Context" instance, initialized by the dns.zone.Zone object.
- Filter-out any validator instances that do not make sense for the DNSSEC type of the zone.
- Run the zone-type validators against the Context.
- Iterate Nodes in the zone object:
- Run the node-type validators against the Context and Node.
- Iterate RRSets in the Node:
- Run the RRSet-type validators against the Context and RRSet.
- Iterate Records in the RRSet:
- Run the record-type validators against the Context and Record.
The use of the dnspython library pervades the application (so if you're familiar with it already, you've got an excellent start): The loaders read from some source and return a dnspython dns.zone.Zone object to the framework. Similarly, the framework presents to the validators the same dns.zone.Zone object for examination.
Zone loaders are classes derived from dns_sprockets_lib.loaders.ZoneLoader (in the dns_sprockets_lib/loaders/__init__.py file), which defines the interface expected by the framework:
class ZoneLoader(object):
'''
[Base class for zone loaders]
'''
LOADER_NAME = None # Automatically set in __init__.
LOADER_OPTARGS = {} # Override possible! e.g.: {'now': (None, 'Time to use for now')}
def __init__(self, args):
'''
Ctor, caches the arguments used to run the application, and grabs any
optional test arguments.
'''
self.LOADER_NAME = utils.camelcase_to_underscores(self.__class__.__name__)
self.args = args
utils.process_optargs(self.LOADER_OPTARGS, self.LOADER_NAME, self)
def run(self):
'''
Runs the zone loader -- must override!
:return: A dns.zone.Zone instance.
'''
pass
Two class variables are expected:
- LOADER_NAME Contains the underscore-style name of the loader, and is automatically set up in the __init__() method.
- LOADER_OPTARGS Contains any plugin-specific parameters that may be set from the command-line ...more on this later.
Two methods are expected:
- __init__() Takes the arguments object containing the command-line options passed by the user to the application.
- run() Invokes the zone loader functionality and returns a dns.zone.Zone object.
As an example, the code for the File loader is show here. It is almost trivial because it takes advantage of the built-in host file loading available in the dnspython library:
class File(loaders.ZoneLoader):
'''
Loads a zone from a file in AXFR-type or Bind host-type format.
'''
LOADER_OPTARGS = {
'rdclass': ('IN', 'Class of records to pull'),
'allow_include': ('1', 'Allow file to include other files')}
def __init__(self, args):
'''
Ctor.
'''
self.rdclass = None
self.allow_include = None
super(File, self).__init__(args)
def run(self):
'''
:return: A dns.zone.Zone instance.
'''
other_args = {
'origin': self.args.zone,
'relativize': False,
'filename': self.args.source,
'check_origin': False,
'rdclass': dns.rdataclass.from_text(self.rdclass),
'allow_include': bool(int(self.allow_include))}
return dns.zone.from_file(self.args.source, **other_args)
Please note the __init__() method calls back into the base class to include its useful and necessary functionality! Also be aware that the class docstring is used for the description of the loader, as shown in the --help output (keep it brief!)
Once the framework obtains a dns.zone.Zone instance from the specified zone loader, it constructs a Context instance from it, which is passed to the validators. In addition to the application's command-line arguments (as context.args) and the actual dns.zone.Zone instance created by the loader (as context.zone_obj), it contains some other attributes for the convenience of validators (code for the Context class can be found in the dns_sprockets_lib/validators/__init__.py file). Some of these are useful to some validators, but can be ignored if not useful:
- context.node_names Contains DNSSEC-ordered list of all node names present in the zone (including empty-non-terminal names implied by wildcard names).
- context.soa_rdataset Contains the zone's SOA RRSet.
- context.dnskey_rdataset Contains the zone's DNSKEY RRSet.
- context.nsec3param_rdataset Contains the zone's NSEC3PARAM RRSet.
- context.delegated_names Contains list of any delegated names in the zone.
- context.dnssec_type Indicates the DNSSEC type of the zone.
A method called is_delegated() is also available, which lets clients easily determine if a given owner name is delegated.
Validators are classes ultimately derived from dns_sprockets_lib.validators._Validator (in the dns_sprockets_lib/validators/__init__.py file). This is the base class for the four more specialized validator classes (ZoneTest, NodeTest, RRSetTest, and RecordTest):
class _Validator(object):
'''
[Base class for validator classes]
'''
TEST_NAME = None # Automatically set in __init__.
TEST_TYPE = None # Override expected! e.g.: ZONE_TEST
TEST_DNSSECTYPE = None # Override possible! one of: None, True, 'NSEC' or 'NSEC3'
TEST_RRTYPE = None # Override possible! e.g.: 'A', or 'RRSIG,NSEC3PARAM'
TEST_OPTARGS = {} # Override possible! e.g.: {'now': (None, 'Time to use for now')}
def __init__(self, args):
'''
Ctor, caches the arguments used to run the application, and grabs any
optional test arguments.
'''
self.TEST_NAME = utils.camelcase_to_underscores(self.__class__.__name__)
self.args = args
utils.process_optargs(self.TEST_OPTARGS, self.TEST_NAME, self)
Five class variables are expected:
- TEST_NAME Contains the underscore-style name of the validator, and is automatically set up in the __init__() method.
- TEST_TYPE Indicates the type of validator.
- TEST_DNSSECTYPE Indicates the DNSSEC-type of the validator.
- TEST_RRTYPE Indicates zero or more resource record types the validator is specialized for. If no types specified, ALL types are assumed.
- TEST_OPTARGS Contains any plugin-specific parameters that may be set from the command-line ...more on this later.
One method is provided:
- __init__() Convenince method for use by sub-classes.
There are four _Validator-derived classes for use by plugins (also defined in the dns_sprockets_lib/validators/__init__.py file). They provide slight convenience by defining TEST_TYPE properly, but more importantly expose different run() signatures, specific to each type of validator:
class ZoneTest(_Validator):
'''
[Base class for zone-type validators]
'''
TEST_TYPE = ZONE_TEST
def run(self, suggested_tested, context):
'''
Runs the zone-type validator.
:param str suggested_tested: A suggested tested value.
:param obj context: The testing context.
:return: A tuple (tested, result)
'''
return ('OOPS!', 'ERROR: run() not overridden for %s' % (self.TEST_NAME))
class NodeTest(_Validator):
'''
[Base class for node-type validators. Derived classes *may* be restricted
to specific RRType's by specifying a TEST_RRTYPE]
'''
TEST_TYPE = NODE_TEST
def run(self, context, suggested_tested, name, node):
'''
Runs the node-type validator. If a TEST_RRTYPE specified, the node
presented to the validator will be filtered accordingly.
:param obj context: The testing context.
:param str suggested_tested: A suggested tested value.
:param str name: The name being tested.
:param obj node: The dns.Node corresponding to the name.
:return: A tuple (tested, result)
'''
return ('OOPS!', 'ERROR: run() not overridden for %s' % (self.TEST_NAME))
class RRSetTest(_Validator):
'''
[Base class for rrset-type validators. Derived classes *may* be restricted
to specific RRType's by specifying a TEST_RRTYPE]
'''
TEST_TYPE = RRSET_TEST
def run(self, context, suggested_tested, name, rdataset):
'''
Runs the name-type validator. If a TEST_RRTYPE is specified, the RRSet
presented to the validator will be filtered accordingly.
:param obj context: The testing context.
:param str suggested_tested: A suggested tested value.
:param str name: The name being tested.
:param obj rdataset: The dns.rdataset corresponding to the name.
:return: A tuple (tested, result)
'''
return ('OOPS!', 'ERROR: run() not overridden for %s' % (self.TEST_NAME))
class RecTest(_Validator):
'''
[Base class for record-type validators. Derived classes *may* be restricted
to specific RRType's by specifying a TEST_RRTYPE]
'''
TEST_TYPE = REC_TEST
def run(self, context, suggested_tested, name, ttl, rdata):
'''
Runs the record-type validator. If a TEST_RRTYPE is specified, the
validator will only see those types of records.
:param obj context: The testing context.
:param str suggested_tested: A suggested tested value.
:param str name: The name of the record being tested.
:param int ttl: The TTL of the record being tested.
:param obj rdata: The dns.rdata.Rdata object being tested.
:return: A tuple (tested, result)
'''
return ('OOPS!', 'ERROR: run() not overridden for %s' % (self.TEST_NAME))
The suggested_tested string contains a default name of the object being tested, be it a zone, node, RRSet or record. It can be used in most instances as the first item in the returned tuple from run():
Notes on the run() return tuple (tested, result): - If a validation is skipped for whatever reason, the 'tested' value should be None, which causes the framework to ignore the run. Otherwise, a value describing the object being tested should be set (and as mentioned 'suggested_tested' is a good value). - The actual result of an un-skipped test is returned in 'result'. If the test passes, simply return None. Otherwise, return a string describing the failure.
As an example, the code for the RrsigTime validator is as follows. The TEST_DNSSECTYPE is set to True to indicate the validation only makes sense for DNSSEC-type zones. It is a record-type test, and only receives RRSIG records due to the TEST_RRTYPE filtering applied. The "context", "name" and "ttl" parameters are ignored for this validation. The "rdata" parameter is used, and is of type dns.rdata.Rdata (a type defined in dnspython):
class RrsigTime(validators.RecTest):
'''
Checks RRSIG's inception <= expiration.
'''
TEST_DNSSECTYPE = True
TEST_RRTYPE = 'RRSIG'
def run(self, context, suggested_tested, name, ttl, rdata):
result = None
if rdata.inception > rdata.expiration:
result = 'Inception time greater than expiration time'
return (suggested_tested, result)
Please note that if your validator needs to define an __init__() method, it must call the base's __init__() to receive its useful and necessary functionality! Also be aware that the class docstring is used for the description of the validator, as shown in the --help output (keep it brief!)
Loaders and validators may have parameters that are specific to themselves. The framework's --define command-line switch is used to pass these parameters to the plugins.
The names of the --define parameters are of the form: <pluginname>_<paramname> (e.g. "rrsig_missing_now" specifies the "now" parameter for the rrsig_missing validator), and are translated and set as plugin attributes as <paramname> (e.g. self.now in rrsig_missing methods).
As an example (and shown earlier), the File zone loader plugin defines two parameters specific to loading zone files:
LOADER_OPTARGS = {
'rdclass': ('IN', 'Class of records to pull'),
'allow_include': ('1', 'Allow file to include other files')}
LOADER_OPTARGS (and TEST_OPTARGS for validators) is a dictionary of parameter descriptors; each entry is keyed by <paramname>, and indexes a 2-tuple of (<defaultvalue>, <description>). If no --define for the parameter is passed, the <defaultvalue> will be set. The <description> is used for --help output, so keep it brief please!