-- Leo's gemini proxy

-- Connecting to going-flying.com:1965...

-- Connected

-- Sending request

-- Meta line: 20 text/gemini; lang=en

[06/14/2021 @13:30]: Sysadmining: DNS Four, Automation


In the previous three DNS posts I covered DNS from a fairly high level and then focused in on some key elements in BIND, the de-facto DNS server package. DNS is such an important part of inter-networking that it is bound to be a bit complex and given that complexity leads towards errors when humans are involved it is a pretty solid use-case for automation.


I mainly use Puppet to automate the state of my systems. I find it to be much more powerful than other tools and it allows me to specify a desired state instead of a set of tasks. It is also one of the most mature automation tools out there and is well supported on the platforms I use.


In general I automate three key pieces of my DNS infrastructure (other than making sure the required packages are installed and up-to-date, of course). 1) Zonefile generation 2) TLSA and DMARC records 3) DNSSEC key rotation and signing


Zonefile generation is easy, they are all generated from templates and concatenated together. This way I can specify records that may change in YAML using Hiera (Puppet's hierarchical key/value store), or resolve them via custom functions (as I do with the TLSA records for example) and have static blocks that are the same for all of my domains (like the NS and MX record sections).


My Puppet master runs various jobs that generate and rotate cryptographic keys. I wrote some custom plugins for LetsEncrypt certbot that lets Puppet manage the certificates and keys. Since they are available to Puppet I can generate the TLSA records that are used for DANE (a method to assure a client that the certificates they are getting from a TLS server are the correct ones). I provide the fingerprints used by the keys in the format the Apache (my webserver of choice) and BIND want using Facter (the Puppet host fact engine). The following Ruby code provides the custom facts to Facter.


# Generate the fingerprint of the SSL cert for things like
# Certificate pinning and TLSA records.
require 'fileutils'
require 'openssl'

Facter.add("ssl_pubkey_fingerprint") do
	confine :kernel => "Linux"

	setcode do
		cert = "/etc/ssl/certs/ssl.ub3rgeek.net_cert.pem"
		fingerprints = {}

		if File.exist?(cert)
			fd = File.read(cert)
			s_cert = OpenSSL::X509::Certificate.new(fd)
			p_key = s_cert.public_key().to_der()
			dgst = OpenSSL::Digest::SHA256.new(p_key)
			fingerprints['ssl.ub3rgeek.net'] = dgst.base64digest()
		end

		cert = "/etc/ssl/certs/going-flying.com_cert.pem"
		if File.exist?(cert)
			fd = File.read(cert)
			s_cert = OpenSSL::X509::Certificate.new(fd)
			p_key = s_cert.public_key().to_der()
			dgst = OpenSSL::Digest::SHA256.new(p_key)
			fingerprints['www.going-flying.com'] = dgst.base64digest()
		end

		fingerprints
	end
end

Facter.add("ssl_pubkey_hexdigest") do
	confine :kernel => "Linux"

	setcode do
		cert = "/etc/ssl/certs/ssl.ub3rgeek.net_cert.pem"
		digests = {}
		if File.exist?(cert)
			fd = File.read(cert)
			s_cert = OpenSSL::X509::Certificate.new(fd)
			p_key = s_cert.public_key().to_der()
			dgst = OpenSSL::Digest::SHA256.new(p_key)
			digests['ssl.ub3rgeek.net'] = dgst.to_s()
		end

		cert = "/etc/ssl/certs/going-flying.com_cert.pem"
		if File.exist?(cert)
			fd = File.read(cert)
			s_cert = OpenSSL::X509::Certificate.new(fd)
			p_key = s_cert.public_key().to_der()
			dgst = OpenSSL::Digest::SHA256.new(p_key)
			digests['www.going-flying.com'] = dgst.to_s()
		end

		digests
	end
end


Finally, the most complex automated task is DNSSEC. DNSSEC is different from DNS over TLS (or DNS over HTTPS). DNSSEC does not seek to provide any privacy, rather it provides a chain of trust back to the root of the global DNS system so your resolver can verify that the answer it is getting is in fact the correct answer. DNSSEC is a huge topic but in general it allows zone administrators to cryptographically sign the records in their zones and any zones that they delegate to, so in going-flying.com you can trace the signatures from going-flying, to com to '.' (the root of DNS) and know for sure that someone isn't spoofing the answer.


I have a script that is called by puppet any time the DNS configuration changes (which is at least monthly due to various cryptographic key rotations and things like LetsEncrypt).


#!/bin/sh
# dnssec-signzones (c) 2017-2018 Matthew J. Ernisse <matt@going-flying.com>
#
# Manage DNSSEC for zones, including rotating keys as needed.

set -e
PATH=$PATH:/sbin:/usr/sbin

# Paths to BIND files
KEYDIR="/etc/bind/keys"
STATEDIR="/var/tmp"
ZONEDIR="/var/cache/bind"

# Timeouts
DEACTIVATE_TIMEOUT="+3d"
DELETE_TIMEOUT="+5d"
PREPUBLISH_INTERVAL="+2d"
REKEY_AGE="30"			# days

# delete any keys that are past their delete-by
delete_old_keys()
{
	local dtime=0
	local now=$(date +"%s")

	for file in $(find ${KEYDIR} -name \*.key); do
		if grep -q 'key-signing' "$file"; then
			continue
		fi

		dtime=$(sed -ne 's/^; Delete:\s*[0-9]*\s*(\(.*\))/\1/p' \
			"$file")

		if [ -z "$dtime" ]; then
			continue
		fi

		dtime=$(date -d "${dtime}" +"%s")
		if [ "$dtime" -le "$now" ]; then
			echo "Deleted expired key: $file"
			rm -- "$file"
			rm -- "$(echo "$file" | sed -e 's/\.key$/.private/')"
		fi
	done
}

find_domains()
{
	for domain in $(ls -1 ${KEYDIR} | \
		sed -ne 's/^K\([-\._a-z0-9]*\)\.\+.*/\1/p' | sort -u); do
		echo $domain
	done
}

# return the latest (currently used) DNSSEC keyfile
get_last_key()
{
	if [ -z $1 ]; then
		echo "usage: get_last_key domain"
		return 127
	fi

	local domain fn m mtime=0
	domain="$1"

	for file in $(find ${KEYDIR} -name K${domain}\*.key); do
		if grep -q 'key-signing' "$file"; then
			continue
		fi

		keyage=$(sed -ne \
		's/^; Activate:\s\{1,\}[0-9]\{1,\}\s\{1,\}(\(.*\))$/\1/p' \
			"$file")
		keyage=$(date -d "$keyage" +%s)

		if [ -z "$keyage" ]; then
			echo "get_last_key() failed to read activation time"
			return 1
		fi

		if [ "$keyage" -gt "$mtime" ]; then
			mtime="$keyage"
			fn="$file"
		fi
	done

	echo "$(basename $fn)"
}

need_new_key()
{
	if [ -z $1 ]; then
		echo "usage: need_new_key domain"
		return 127
	fi

	local domain last_key keyage now
	domain="$1"
	now=$(date +%s)

	last_key=$(get_last_key "$domain")
	keyage=$(sed -ne \
		's/^; Activate:\s\{1,\}[0-9]\{1,\}\s\{1,\}(\(.*\))$/\1/p' \
		"$KEYDIR/$last_key")
	keyage=$(date -d "$keyage" +%s)

	if [ $(( $now - $keyage )) -ge $(( ${REKEY_AGE} * 86400 )) ]; then
		echo "key ${last_key} needs rotating"
		return 0
	fi

	return 1
}

rotate_keys()
{
	if [ -z $1 ]; then
		echo "usage: rotate_keys domain"
		return 127
	fi

	domain="$1"
	last_key=$(get_last_key "$domain")

	# Logically this should go the other way, generate a new and then
	# if successful expire the old but dnssec-keygen refuses to generate
	# a successor key if there is no expire time on the current key..
	echo "Setting expire times on ${last_key}"
	if ! dnssec-settime -K "$KEYDIR" -I "$DEACTIVATE_TIMEOUT" \
		-D "$DELETE_TIMEOUT" "$last_key"; then
		exit 1
	fi

	echo "Generating new key..."
	if ! dnssec-keygen -K "$KEYDIR" -S "$last_key" \
		-i "$PREPUBLISH_INTERVAL" -q; then
		exit 1
	fi

}

sign_domain()
{
	if [ -z $1 ]; then
		echo "usage: sign_domain domain"
		return 127
	fi

	domain="$1"

	for view in "public" "private"; do
		zonefile="${view}-${domain}.hosts"

		if [ -f "$ZONEDIR/$zonefile" ]; then
			sign_zone "$domain" "$zonefile"
		fi
	done
}

sign_zone()
{
	if [ -z "$1" ] || [ -z "$2" ]; then
		echo "usage: sign_zone domain zonefile"
		return 127
	fi

	domain="$1"
	zonefile="$2"

	echo "signing $zonefile for $domain"
	dnssec-signzone -S -K "$KEYDIR" -o "$domain" \
		"$ZONEDIR/$zonefile"
}

# update the zone's serial -- this is no longer handled by puppet.
update_serial()
{
	if [ -z $1 ]; then
		echo "usage: update_serial domain"
		return 127
	fi
	local new_serial old_rev old_rev_stripped old_serial=0
	local zonefile

	if [ -f "${STATEDIR}/${domain}.serial" ]; then
		old_serial=$(< "${STATEDIR}/${domain}.serial")
		old_rev=$(echo "$old_serial" | sed -ne \
			's/^[0-9]\{,8\}\([0-9]\{,2\}\)$/\1/p')
		old_serial=$(echo "$old_serial" | sed -ne \
			's/^\([0-9]\{,8\}\).*/\1/p')

		# Bash will think we're in octal if we feed it a number
		# with a leading zero.
		old_rev_stripped=$(echo "$old_rev" | sed -ne \
			's/^0\([0-9]\)/\1/p')
		if [ -n "$old_rev_stripped" ]; then
			old_rev="$old_rev_stripped"
		fi
	fi

	new_serial="$(date +%Y%m%d)"

	if [ "$new_serial" -eq "$old_serial" ]; then
		if [ $(( $old_rev + 1 )) -lt 10 ]; then
			new_serial="${new_serial}0$(( $old_rev + 1 ))"
		else
			new_serial="${new_serial}$(( $old_rev + 1 ))"
		fi
	else
		new_serial="${new_serial}00"
	fi

	echo "$new_serial" > "${STATEDIR}/${domain}.serial"

	for view in "public" "private"; do
		zonefile="${ZONEDIR}/${view}-${domain}.hosts"
		sed -e "s/##SERIAL_NUMBER##/${new_serial}/" \
			"${zonefile}.puppet" > "$zonefile"
	done
}

for domain in $(find_domains); do
	if need_new_key "$domain"; then
		rotate_keys "$domain"
	fi

	delete_old_keys
	update_serial "$domain"
	sign_domain "$domain"
done


The script is complex because it needs to do several things but the for loop at the bottom tells the story of what is happening. The primary job of the script is to manage the DNSSEC keys, rotating them and expiring old ones, so since this script is run every time Puppet every time any of the DNS zonefiles are changed the first thing we do is to check to see if we actually need to rotate the keys. Once we do that we simply expire any old keys, update the serial numbers on the zones and then sign them. I can't imagine this script working verbatim for anyone else, it assumes a lot of things that are probably only true in my setup, but it should get anyone looking to automate DNSSEC with BIND well on their way.


DNS is one of the most important services provided on the Internet. It is used every time you type a name instead of an IP address. It is used for service discovery and configuration, e-mail SPAM detection, malware prevention and is the core method CDNs use to steer you to their closest edge location to optimize delivery of everything from JPEGs to critical software patches. Being comfortable with it gives a systems / network administrator tremendous control over network traffic and performance, it is an extremely powerful tool in the security administrator's portfolio, being able to short circuit command and control server connections and prevent malicious payloads from being downloaded, and any support people as trouble in DNS land is the root cause of so many end-user problems. I highly recommend _DNS and BIND_, by Cricket Liu and Paul Albitz, published by O'Reilly Media as further reading.





↩ back to index

backlinks [GUS]

backlinks [geminispace.info]


🚀 © MMXX-MMXXI matt@going-flying.com

-- Response ended

-- Page fetched on Tue Sep 21 08:39:39 2021