ADUFRAY

I recently upgraded my AT&T modem from the NVG 589 to the Pace 5268AC at the insistence of a technician out to repair my ONT. I had been having some packet loss issues and hoped the upgrade would help (it seemed to). However, I noticed immediately my NTP packets were being filtered. I thought this was odd, but found the following resources:

After studying all of these pages and looking at countless lines of tcpdump output, I confirmed a few things:

Sadly, the Network Time Foundation’s reference implementation of ntpd only permits using an ephemeral source port when using ntpdate and not from ntpd itself. Therefore, while behind an AT&T network, it seems not possible to run ntpd without some kind of middleware piece to mangle your UDP packets (e.g., iptables). This makes things very difficult in a modern home with all the following devices using NTP:

I was able to configure a local NTP server using OpenBSD’s OpenNTPd, which uses ephemeral source ports by default. Most of my devices I was able to reconfigure to use my local time server, however many devices do not expose that configuration to the end user - in particular iOS devices and Apple TVs. In fact, they were the noisiest devices on the network attempting to sync time. In order to resolve those devices, I had to add stub DNS zones to my local DNS. I created replacement zones for:

Once I pointed those hostnames to my local NTP server, everything seemed to resolve itself. Even systems that I cannot change from ntpd to OpenNTPd can now sync time using my local servers, which is pretty great.

I also looked at using Chrony instead of OpenNTPd, but after installation on my FreeBSD server I received this scary message:

Unfortunately, this software has shameful history of several vulnerabilities
previously discovered.  FreeBSD Project cannot guarantee that this spree had
come to an end.  Please type ``pkg delete chrony'' to deinstall the port
if tight security is a concern.

I liked the idea of using Chrony because my system monitoring tool of choice, Check_MK, already had a plugin to monitor Chrony, and as far as I can tell didn’t have one for OpenNTPd. On top of that, Chrony is now the default NTP server in Red Hat Enterprise Linux starting with version 7.

Since I trust OpenBSD’s security track record vs. a group I’ve never heard of and the most alarming message I’ve ever seen installing a port, I stuck with OpenNTPd. I was also able to make quick work of a Check_MK plugin, despite ntpctl’s absolutely terrible output. (Seriously, who designed that?) Here’s the plugin - you just need to add the sample output’s command either in your plugins directory or directly to check_mk_agent itself, prepending it with <<<openntpd>>> of course.

#!/usr/bin/python

# 2017 (c) adufray.com
# bsd license
# openntpd check_mk plugin to fit with ntp checks
# note: only supports servers/peers, not sensors

ntp_default_levels = (10, 200.0, 500.0) # stratum, ms offset

# Example output from agent:
# $ ntpctl -s a | paste - - | nl | egrep -E '^     1[[:space:]]|[*]' | cut -f 2- | sed -e 's/[[:space:]][[:space:]]*/ /g'
# 4/4 peers valid, clock synced, stratum 4
# 69.164.202.202 from pool us.pool.ntp.org * 1 10 3 2s 30s -0.053ms 8.154ms 1.675ms

def inventory_openntpd(info):
    if info[0]:
        return [(None, "ntp_default_levels")]

def check_openntpd(_no_item, params, info):
    if not info[0]:
        yield 2, "No status information, openntpd probably not running"
        return
    if "clock unsynced" in " ".join(info[0]):
        yield 2, "%s" % " ".join(info[0])
        return

    # Prepare parameters
    crit_stratum, warn, crit = params

    # Check offset and stratum, output a few info texsts
    offset = float(info[1][-3][0:-2])
    stratum = int(info[1][-6])

    # Check stratum
    infotext = "stratum %d" % stratum
    if stratum >= crit_stratum:
        yield 2, infotext + " (maximum allowed is %d)" % (crit_stratum - 1)
    else:
        yield 0, infotext

    # Check offset
    status = 0
    infotext = "offset %.4f ms" % offset
    if abs(offset) >= crit:
        status = 2
    elif abs(offset) >= warn:
        status = 1
    if status:
        infotext += " (levels at %.4f/%.4f ms)" % (warn, crit)
    yield status, infotext, [ ("offset", offset, warn, crit, 0, None) ]

    # Show additional information
    if info[1][1] == "from":
       yield 0, "reference: %s" % "/".join(info[1][0:4:3])
    else:
       yield 0, "reference: %s" % "/".join(info[1][0:2])

check_info["openntpd"] = {
   'check_function':          check_openntpd,
   'inventory_function':      inventory_openntpd,
   'service_description':     'NTP Time',
   'has_perfdata':            True,
   'group':                   'ntp_time',
}

I suppose this will be a solid enough solution until Poul-Henning Kamp’s masterpiece, Ntimed, is released!