GPSD Time Service HOWTO

GPSD, NTP and a GPS receiver supplying 1PPS (one pulse-per-second) output can be used to set up a high-quality NTP time server. This HOWTO explains the method and various options you have in setting it up.

Here is the quick-start sequence. The rest of this document goes into more detail about the steps.

This document does not attempt to explain all the intricacies of time service; it is focused on practical advice for one specific deployment case. There is an introduction [TIME-INTRO] to basic concepts and terminology for those new to time service. An overview of the NTP protocols can be found at [WIKI-NTP], and the official NTP FAQ [NTP-FAQ] is probably as gentle an introduction to the NTP reference implementation as presently exists.

We encourage others to contribute additions and corrections.

There are a few important terms we need to define up front. Latency is delay from a time measurement until a report on it arrives where it is needed. Jitter is short-term variation in latency. Wobble is a jitter-like variation that is long compared to typical measurement periods. Accuracy is the traceable offset from 'true' time as defined by a national standard institute.

A good analogy to jitter vs wobble is changes in sea level on a beach. Jitter is caused by wave action, wobble is the daily effect of tides. For a time server, the most common causes of wobble are varying GPS satellite geometries and the effect of daily temperature variations on the oscillators in your equipment.

See [TIME-INTRO] for a technical description of how NTP corrects your computer’s clock against wobble. For purposes of this how-to, the important concepts to take way are those of time strata, servers, and reference clocks.

Ordinary NTP client computers are normally configured to get time from one or more Stratum 2 (or less commonly Stratum 3) NTP servers. However, with GPSD and a suitable GPS receiver, you can easily condition your clock to higher accuracy than what you get from typical Stratum 2; with a little effort, you can do better than you can get from most public Stratum 1 servers.

You can then make your high-quality time available to other systems on your network, or even run a public NTP server. Anyone can do this; there is no official authority, and any NTP client may choose to use your host as a server by requesting time from it. The time-service network is self-regulating, with NTP daemons constantly pruning statistical outliers so the timebase cannot be accidentally or deliberately compromised.

In fact many public and widely-trusted Stratum 1 servers use GPS receivers as their reference clocks, and a significant fraction of those use GPSD in the way we will describe here.

The way time is shipped from GPS satellites causes problems to beware of in certain edge cases.

Date and time in GPS is represented as number of weeks from the start of zero second of 6 January 1980, plus number of seconds into the week. GPS time is not leap-second corrected, though satellites also broadcast a current leap-second correction which is updated on six-month boundaries according to rotational bulletins issued by the International Earth Rotation and Reference Systems Service (IERS).

The leap-second correction is only included in the multiplexed satellite subframe broadcast, once every 12.5 minutes. While the satellites do notify GPSes of upcoming leap-seconds, this notification is not necessarily processed correctly on consumer-grade devices, and may not be available at all when a GPS receiver has just cold-booted. Thus, reported UTC time may be slightly inaccurate between a cold boot or leap second and the following subframe broadcast.

GPS date and time are subject to a rollover problem in the 10-bit week number counter, which will re-zero every 1024 weeks (roughly every 19.6 years). The first rollover since GPS went live in 1980 was in Aug-1999, followed by Apr-2019, the next will be in Nov-2038 (the 32-bit and POSIX issues will probably be more important by then). The new "CNAV" data format extends the week number to 13 bits, with the first rollover occurring in Jan-2137, but this is only used with some newly added GPS signals, and is unlikely to be usable in most consumer-grade receivers currently.

For accurate time reporting, therefore, a GPS requires a supplemental time references sufficient to identify the current rollover period, e.g. accurate to within 512 weeks. Many GPSes have a wired-in (and undocumented) assumption about the UTC time of the last rollover and will thus report incorrect times outside the rollover period they were designed in.

For accurate time service via GPSD, you require three things:

GPSD is useful for precision time service because it can use the 1PPS pulse delivered by some GPS receivers to discipline (correct) a local NTP instance.

It’s tempting to think one could use a GPS receiver for time service just by timestamping the arrival of the first character in the report on each fix and correcting for a relatively small fixed latency composed of fix-processing and RS232 transmission time.

At one character per ten bits (counting framing and stopbits) a 9600-bps serial link introduces about a mSec of latency per character; furthermore, your kernel will normally delay delivery of characters to your application until the next timer tick, about every 4 mSec in modern kernels. Both USB and RS232 will incur that approximately 5mSec-per-char latency overhead. You’ll have to deal with this latency even on chips like the Venus 6 that claim the beginning of their reporting burst is synced to PPS. (Such claims are not always reliable, in any case.)

Unfortunately, fix reports are also delayed in the receiver and on the link by as much as several hundred mSec, and this delay is not constant. This latency varies (wobbles) throughout the day. It may be stable to 10 mSec for hours and then jump by 200mSec. Under these circumstances you can’t expect accuracy to UTC much better than 1 second from this method.

For example: SiRF receivers, the make currently most popular in consumer-grade GPS receivers, exhibit a wobble of about 170mSec in the offset between actual top-of-second and the transmission of the first sentence in each reporting cycle.

To get accurate time, then, the in-band fix report from the GPS receiver needs to be supplemented with an out-of-band signal that has a low and constant or near-constant latency with respect to the time of the fix. GPS satellites deliver a top-of-GPS-second notification that is nominally accurate to 50nSec; in capable GPS receivers that becomes the 1PPS signal.

1PPS-capable GPS receivers use an RS-232 control line to ship the 1PPS edge of second to the host system (usually Carrier Detect or Ring Indicator; GPSD will quietly accept either). Satellite top-of-second loses some accuracy on the way down due mainly to variable delays in the ionosphere; processing overhead in the GPS receiver itself adds a bit more latency, and your local host detecting that pulse adds still more latency and jitter. But it’s still often accurate to on the order of 1 uSec.

Under most Unixes there are two ways to watch 1PPS; Kernel PPS (KPPS) and plain PPS latching. KPPS is an implementation of RFC 2783 [RFC-2783]. Plain PPS just references the pulse to the system clock as measured in user space. These have different error budgets.

Kernel PPS uses a kernel function to accurately timestamp the status change on the PPS line. Plain PPS has the kernel wake up the GPSD PPS thread and then the PPS thread reads the current system clock. As noted in the GPSD code, having the kernel do the time stamp yields lower latency and less jitter. Both methods have accuracy degraded by interrupt-processing latency in the kernel serial layer, but plain PPS incurs additional context-switching overhead that KPPS does not.

With KPPS it is very doable to get the system clock stable to ±1 uSec. Otherwise, you are lucky to get ±5 uSec, and there will be about 20uSec of jitter. All these figures were observed on plain-vanilla x86 PCs with clock speeds in the 2GHz range.

All the previous figures assume you’re using PPS delivered over RS232. USB GPS receivers that deliver 1PPS are rare, but do exist. Notably, there’s the Navisys GR-601W/GR-701W/GR-801W [MACX-1]. In case these devices go out of production it’s worth noting that they are a trivial modification of the stock two-chip-on-a-miniboard commodity-GPS-receiver design of engine plus USB-to-serial adapter; the GR-[678]01W wires a u-blox 6/7/8 to a Prolific Logic PL23203. To get 1PPS out, this design just wires the 1PPS pin from the GPS engine to the Carrier Detect pin on the USB adapter. (This is known as the "Macx-1 mod".)

With this design, 1PPS from the engine will turn into a USB event that becomes visible to the host system (and GPSD) the next time the USB device is polled. USB 1.1 polls 1024 slots every second. Each slot is polled in the same order every second. When a device is added it is assigned to one of those 1024 polling slots. It should then be clear that the accuracy of a USB 1.1 connected GPS receiver would be about 1 mSec.

As of mid-2016 no USB GPS receiver we know of implements the higher polling-rate options in USB 2 and 3 or the interrupt capability in USB 3. When one does, and if it has the Macx-1 mod, higher USB accuracy will ensue.

Observed variations from the typical figure increase towards the bottom of the table. Notably, a heavily loaded host system can reduce PPS accuracy further, though not KPPS accuracy except in the most extreme cases. The USB poll interval tends to be very stable (relative to its 1mSec or 100μSec base).

Network NTP time accuracy can be degraded below RFC5905’s "a few tens of milliseconds" by a number of factors. Almost all have more to do with the quality of your Internet connection to your servers than with the time accuracy of the servers themselves. Some negatives:

With these factors in play, worst-case error can reach up to ±100mSec. Fortunately, errors of over ±100mSec are unusual and should occur only if all your network routes to servers have serious problems.

If your kernel provides the RFC 2783 KPPS (kernel PPS) API, gpsd will use that for extra accuracy. Many Linux distributions have a package called "pps-tools" that will install KPPS support and the timepps.h header file. We recommend you do that. If your kernel is built in the normal modular way, this package installation will suffice.

To get 1PPS to your NTP daemon, you first need to get it from a PPS-capable GPS receiver. As of early 2015 this means either the previously mentioned GR devices or a serial GPS receiver with 1PPS.

You can find 1PPS-capable devices supported by GPSD at [HARDWARE]. Note that the most popular consumer-grade GPS receivers do not usually deliver 1PPS through USB or even RS232. The usual run of cheap GPS mice won’t do. In general, you can’t use a USB device for time service unless you know it has the Macx-1 mod.

In the past, the RS232 variant of the Garmin GPS-18 has been very commonly used for time service (see [LVC] for a typical setup very well described). While it is still a respectable choice, newer devices have better sensitivity and signal discrimination. This makes them superior for indoor use as time sources.

In general, use a GPS receiver with an RS232 interface for time service if you can. The GR-601W was designed (by one of the authors, as it happens) for deployment with commodity TCP/IP routers that only have USB ports. RS232 is more fiddly to set up (with older devices like the GPS-18 you may even have to make your own cables) but it can deliver three orders of magnitude better accuracy and repeatability — enough to meet prevailing standards for a public Stratum 1 server.

Among newer receiver designs the authors found the u-blox line of receivers used in the GR-[678]01W to be particularly good. Very detailed information on its timing performance can be found at [UBLOX-TIMING]. One of us (Raymond) has recent experience with an eval kit, the EVK 6H-0-001, that would make an excellent Stratum 0 device.

Both the EVK 6H and GR-601W are built around the LEA-6H module, which is a relatively inexpensive but high-quality navigation GPS receiver. We make a note of this because u-blox also has a specialized timing variant, the LEA 6T, which would probably be overkill for an NTP server. (The 6T does have the virtue that you could probably get a good fix from one satellite in view once it knows its location, but the part is expensive and difficult to find.)

Unfortunately as of early 2015 the LEA-6H is still hard to find in a packaged RS232 version, as opposed to a bare OEM module exporting TTL levels or an eval kit like the EVK 6H-0-001 costing upwards of US$300. Search the web; you may find a here-today-gone-tomorrow offer on alibaba.com or somewhere similar.

The LEA-6T, and some other higher-end GPS receivers (but not the LEA-6H) have a stationary mode which, after you initialize it with the device’s location, can deliver time service with only one good satellite lock (as opposed to the three required for a fix in its normal mode). For most reliable service we recommend using stationary mode if your device has it. GPSD tools don’t yet directly support this, but that capability may be added in a future release.

The design of your host system can also affect time quality. The ±5uSec error bound quoted above is for a dual-core or better system with clock in the 2GHz range on which the OS can schedule the long-running PPS thread in GPSD on an otherwise mostly unused processor (the Linux scheduler, in particular, will do this). On a single-core system, contention with other processes can pile on several additional microseconds of error.

If you are super-serious about your time-nuttery, you may want to look into the newest generation of dedicated Stratum 1 microservers being built out of open-source SBCs like the Raspberry Pi and Beaglebone, or sometimes with fully custom designs. A representative build is well described at [RPI].

These microserver designs avoid load-induced jitter by being fully dedicated to NTP service. They are small, low-powered devices and often surprisingly inexpensive, as in costing less than US$100. They tend to favor the LEA-6H, and many of them use preinstalled GPSD on board.

You can determine whether your GPS receiver emits 1PPS, and gpsd is detecting it, by running the gpsmon utility (giving it the GPS receiver’s serial-device path as argument). Watch for lines of dashes marked 'PPS' in the packet-logging window; for most GPS receiver types there will also be a "PPS offset:" field in the data panels above showing the delta between PPS and your local clock.

If you don’t have gpsmon available, or you don’t see PPS lines in it, you can run ppscheck. As a last resort you can gpsd at -D 5 and watch for PPS state change messages in the logfile.

If you don’t see evidence of incoming PPS, here are some trouble sources to check:

If you’re going to use gpsd for time service, you must run in -n mode so the clock will be updated even when no clients are active. This option is forced if you built GPSD with timeservice=yes as an option.

Note that gpsd assumes that after each fix the GPS receiver will assert 1PPS first and ship sentences reporting time of fix second (and the sentence burst will end before the next 1PPS). Every GPS we know of does things in this order. (However, on some very old GPSes that defaulted to 4800 baud, long sentence bursts — notably those containing a skyview — could slop over into the next second.)

If you ever encounter an exception, it should manifest as reported times that look like they’re from the future and require a negative fudge. If this ever happens, please report the device make and model to the GPSD maintainers, so we can flag it in our GPS hardware database.

There is another possible cause of small negative offsets which shows up on the GR-601W: implementation bugs in your USB driver, combining with quantization by the USB poll interval. This doesn’t mean the u-blox 6 inside it is actually emitting PPS after the GPS timestamp is shipped.

In order to present the smallest possible attack surface to privilege-escalation attempts, gpsd, if run as root, drops its root privileges very soon after startup - just after it has opened any serial device paths passed on the command line.

Thus, KPPS can only be used with devices passed that way, not with GPSes that are later presented to gpsd by the hotplug system. Those hotplug devices may, however, be able to use plain, non-kernel PPS. gpsd tries to automatically fall back to this when absence of root permissions makes KPPS unavailable.

In general, if you start gpsd as other than root, the following things will happen that degrade the accuracy of reported time:

You may also find gpsd can’t open serial devices at all if your OS distribution has done "secure" things with the permissions.

Most Unix systems get their time service through ntpd, a very old and stable open-source software suite which is the reference implementation of NTP. The project home page is [NTP.ORG]. We recommend using NTPsec, a recent fork that is improved and security-hardened [NTPSEC.ORG].

When gpsd receives a sentence with a timestamp, it packages the received timestamp with current local time and sends it to a shared-memory segment with an ID known to ntpd, the network time synchronization daemon. If ntpd has been properly configured to receive this message, it will be used to correct the system clock.

When in doubt, the preferred method to start your timekeeping is:

where /dev/ttyXX is whatever 1PPS-capable device you have. In a binary-package-based Linux distribution it is probable that ntpd will already have been launched at boot time.

It’s best to have gpsd start first. That way when ntpd restarts it has a good local time handy. If ntpd starts first, it will set the local clock using a remote, probably pool, server. Then ntpd has to spend a whole day slowly resyncing the clock.

If you’re using dhcp3-client to configure your system, make sure you disable /etc/dhcp3/dhclient-exit-hooks.d/ntp, as dhclient would restart ntpd with an automatically created ntp.conf otherwise — and gpsd would not be able to talk with ntpd anymore.

While gpsd may be runnable as non-root, you will get significantly better accuracy of time reporting in root mode; the difference, while almost certainly insignificant for feeding Stratum 1 time to clients over the Internet, may matter for PTP service over a LAN. Typically only root can access kernel PPS, whereas in non-root mode you’re limited to plain PPS (if that feature is available). As noted in the previous section on 1PPS quality issues, this difference has performance implications.

The rest of these setup instructions will assume that you are starting gpsd as root, with occasional glances at the non-root case.

Now check to see if gpsd has correctly attached the shared-memory segments in needs to communicate with ntpd. ntpd’s rules for the creation of these segments are:

Because gpsd can be started either as root or non-root, it checks and attaches the more privileged segment pair it can — either 0 and 1 or 2 and 3.

For each GPS receiver that gpsd controls, it will use the attached ntpshm segments in pairs (for coarse clock and pps source, respectively) starting from the first found segments.

To debug, try looking at the live segments this way:

If gpsd was started as root, the results should look like this:

For a bit more data try this:

If gpsd cannot open the segments, check that you are not running SELinux or apparmor. Either may require you to configure a security exception.

If you see the shared segments (keys 1314148400 — 1314148403), and no gpsd or ntpd is running then try removing them like this:

Here is a minimal sample ntp.conf configuration to work with GPSD run as root, telling ntpd how to read the GPS notifications

The number "0.9999" is a placeholder, to be explained shortly. It is not a number to be used in production - it’s too large. If you can’t replace it with a real value, it would be best to leave out the clause entirely so the entry looks like:

This is equivalent to declaring a time1 of 0.

The pool statement adds a variable number of servers (often 10) as additional time references needed by ntpd for redundancy and to give you a reference to see how well your local GPS receiver is performing. If you are outside of the USA replace the pool servers with one in your local area. See [USE-POOL] for further information.

The pool statement, and the driftfile and logfile declarations after it, will not be strictly necessary if the default ntp.conf that your distribution supplies gives you a working setup. The two pairs of server and fudge declarations are the key.

ntpd can be used in Denial of Service (DoS) attacks. To prevent that, but still allow clients to request the local time, be sure the 'restrict' statements are in your ntpd config file. For more information see [CVE-2009-3563].

Users of ntpd versions older than revision ntp-4.2.5p138 should instead use this ntp.conf, when gpsd is started as root:

Users of ntpd versions prior to ntp-4.2.5 do not have the "pool" option. Alternative configurations exist, but it is recommended that you upgrade ntpd, if feasible.

The magic pseudo-IP address 127.127.28.0 identifies unit 0 of the ntpd shared-memory driver (NTP0); 127.127.28.1 identifies unit 1 (NTP1). Unit 0 is used for in-band message timestamps and unit 1 for the (more accurate, when available) time derived from combining in-band message timestamps with the out-of-band PPS synchronization pulse. Splitting these notifications allows ntpd to use its normal heuristics to weight them.

Different units — 2 (NTP2) and 3 (NTP3), respectively — must be used when gpsd is not started as root. Some GPS HATs put PPS time on a GPIO pin and will also use unit 2 (NTP2) for the PPS time correction.

With this configuration, ntpd will read the timestamp posted by gpsd every 64 seconds (16 if non-root) and send it to unit 0.

The number after the parameter time1 (0.9999 in the example above) is a "fudge", offset in seconds. It’s an estimate of the latency between the time source and the 'real' time. You can use it to compensate out some of the fixed delays in the system. An 0.9999 fudge would be ridiculously large.

You may be able to find a value for the fudge by looking at the entry for your GPS receiver type on [HARDWARE]. Later in this document we’ll explain methods for estimating a fudge factor on unknown hardware.

There is nothing magic about the refid fields; they are just labels used for generating reports. You can name them anything you like.

When you start gpsd, it will wait for a few good fixes before attempting to process PPS. You should run cgps to verify your GPS receiver has a 3D lock before worrying about timekeeping.

After starting (as root) ntpd, then gpsd, a listing similar to the one below should appear as the output of the command "ntpq -p" (after allowing the GPS receiver to acquire a 3D fix). This may take up to 30 minutes if your GPS receiver has to cold-start or has a poor skyview.

The line with refid ".GPS." represents the in-band time reports from your GPS receiver. When you are getting PPS then it may look like this:

Note the additional ".PPS." line.

If the value under "reach" for the SHM lines remains zero, check that gpsd is running; cgps reports a 3D fix; and the '-n' option was used. Some GNSS receivers specialized for time service can report time with signal lock on only one satellite, but with most devices a 3D fix is required.

When the SHM(0) line does not appear at all, check your ntp.conf and the system logs for error messages from ntpd.

Notice the 1st and 3rd servers, stratum 1 servers, disagree by more than 8 mSec. The 1st and 2nd servers disagree by over 12 mSec. Our local PPS reference agrees to the clock.sjc.he.net server within the expected jitter of the GR-601W in use.

When no other servers or local reference clocks appear in the NTP configuration, the system clock will lock onto the GPS clock, but this is a fragile setup — you can lose your only time reference if the GPS receiver is temporarily unable to get satellite lock.

You should always have at least two (preferably four) fallback servers in your ntpd.conf for proper ntpd operation, in case your GPS receiver fails to report time. The 'pool' command makes this happen. And you’ll need to adjust the offsets (fudges) in your ntp.conf so the SHM(0) time is consistent with your other servers (and other local reference clocks, if you have any). We’ll describe how to diagnose and tune your server configuration in a later section.

NTP is a cluster protocol. It sorts time sources into clusters, picks the best cluster, then the best source in that cluster, for its time source. By default, NTPsec requires 3 "sane" time sources before serving time. This is set with the minsane option. Not at all recommended, but you can force ntpd to rely sole on your PPS source by adjusting minsane like this in your ntp.conf:

Also note that after cold-starting ntpd it may calibrate for up to 15 minutes before it starts adjusting the clock. Because the frequency error estimate ("drift") that NTP uses isn’t right when you first start NTP, there will be a phase error that persists while the frequency is estimated. So if your clock is a little slow, then it will keep getting behind, and the positive offset will cause NTP to adjust the phase forward and also increase the frequency offset error. After a day or so or maybe less the frequency estimate will be very close and there won’t be a persistent offset.

The GPSD developers would like to receive information about the offsets (fudges) observed by users for each type of receiver. If your GPS receiver is not present in [HARDWARE], or doesn’t have a recommended fudge, or you see a fudge value very different from what’s there, send us the output of the "ntpq -p" command and the make and type of receiver.

chrony is an alternative open-source implementation of NTP service, originally designed for systems with low-bandwidth or intermittent TCP/IP service. It interoperates with ntpd using the same NTP protocols. Unlike ntpd which is designed to always be connected to multiple internet time sources, chrony is designed for long periods of offline use. Like ntpd, it can either operate purely as a client or provide time service. The chrony project has a home page at [CHRONY]. Its documentation includes an instructive feature comparison with ntpd at [CHRONY-COMPARE].

gpsd, when run as root, may feed time information to chronyd using sockets. The sockets are named /run/chrony.XXXX.sock. Where XXXX is replaced by the basenames of the device names gpsd is using. If your receiver outputs serial data on /dev/ttyS0, then the corresponding socket is /run/chrony.ttyS0.sock. If your PPS is on /dev/pps0, then the corresponding socket is /run/chrony.pps0.sock.

Older systems may use the /var/run directory instead of /run. If gpsd can not open the sockets there, it falls back to try /tmp.

No gpsd configuration is required to talk to chronyd. chronyd is configured using the file /etc/chrony.conf or /etc/chrony/chrony.conf. Check your distributions documentation for the correct location.

To get chronyd to connect to gpsd using the socket method add the following lines your chrony.conf file. Except, replace XXXX with the basename of your device’s serial port, often ttyS0, ttyACM0, or ttyAMA0. Replace YYYY with the basename of your PPS device, usually pps0.

When running as root:

Older systems may use the /var/run directory for the socket file instead of /run. /run is compliant with FHS 3.0. If that file can not be opened then gpsd falls back to trying in /tmp:

You can also get gpsd to connect to chronyd using the basic ntpd compatible SHM method. To use that instead of sockets, add these lines to the basic chrony.conf file:

You need to add the "precision 1e-7" on the SHM 1 line as chronyd fails to read the precision from the SHM structure. Without knowing the high precision of the PPS on SHM 1 it may not place enough importance on its data.

If you are outside of the USA replace the pool servers with one in your local area. See [USE-POOL] for further information.

The offset option is functionally like ntpd’s time1 option, used to correct known and constant latency.

The 'allow' option allows anyone on the internet to query your server’s time.

See the chrony man page for more detail on the configuration options [CHRONY-MAN].

Finally note that chronyd needs to be started before gpsd so the sockets are ready before gpsd starts up.

If running as root, the preferred starting procedure is:

After you have verified with cgps that your GPS receiver has a good 3D lock you can check that gpsd is outputting good time by running ntpshmmon.

If you see only "NTP2", instead, you forgot to go root before starting gpsd.

Once ntpshmmon shows good time data you can see how chrony is doing by running 'chronyc sources'. Your output will look like this:

The stratum is as in ntpq. The Poll is how many seconds elapse between samples. The Reach is as in ntpq. LastRx is the time since the last successful sample. Last sample is the offset and jitter of the source.

To keep chronyd from preferring NMEA time over PPS time, you can add an overlarge fudge to the NMEA time. Or add the suffix 'noselect' so it is never used, just monitored.

This section is general and can be used with either ntpd or chronyd. We’ll have more to say about tuning techniques for the specific implementations in later sections.

The clock crystals used in consumer electronics have two properties we are interested in: accuracy and stability. Accuracy is how well the measured frequency matches the number printed on the can. Stability is how well the frequency stays the same even if it isn’t accurate. (Long term aging is a third property that is interesting, but ntpd and chrony both a use a drift history that is relatively short; thus, this is not a significant cause of error.)

Typical specs for oscillator packages are 20, 50, 100 ppm. That includes everything; initial accuracy, temperature, supply voltage, aging, etc.

With a bit of software, you can correct for the inaccuracy. ntpd and chrony both call it drift. It just takes some extra low order bits on the arithmetic doing the time calculations. In the simplest case, if you thought you had a 100 MHz crystal, you need to change that to something like 100.000324. The use of a PPS signal from gpsd contributes directly to this measurement.

Note that a low drift contributes to stability, not necessarily accuracy.

The major source of instability is temperature. Ballpark is the drift changes by 1 PPM per degree C. This means that the drift does not stay constant, it may vary with a daily and yearly pattern. This is why the value of drift the ntpd uses is calculated over a (relatively) short time.

So how do we calculate the drift? The general idea is simple. Measure the time offset every N seconds over a longer window of time T, plot the graph, and fit a straight line. The slope of that line is the drift. The units cancel out. Parts-per-million is a handy scale.

How do you turn that hand waving description into code? One easy way is to set N=2 and pick the right T, then run the answer through a low pass filter. In that context, there are two competing sources of error. If T is too small, the errors on each individual measurement of the offset time will dominate. If T is too big, the actual drift will change while you are measuring it. In the middle is a sweet spot. (For an example, see [ADEV-PLOT].)

Both ntpd and chrony use this technique; ntpd also uses a more esoteric form of estimation called a "PLL/FLL hybrid loop". How T and N are chosen is beyond the scope of this HOWTO and varies by implementation and tuning parameters.

If you turn on the right logging level ("statistics loopstats peerstats" for ntpd, "log measurements tracking" for chronyd), that will record both offset, drift, and the polling interval. The ntpd stats are easy to feed to gnuplot, see the example script in the GPSD contrib directory. The most important value is the offset reported in the 3rd field in loopstats and the last field in tracking.log. With gnuplot you can compare them (after concatenating the rotated logs):

While your NTP daemon (ntpd or chrony) is adjusting the polling interval, it is assuming that the drift is not changing. It gets confused if your drift changes abruptly, say because you started some big chunk of work on a machine that’s usually idle and that raises the temperature.

Your NTP daemon writes out the drift every hour or so. (Less often if it hasn’t changed much to reduce the workload on flash file systems.) On startup, it reloads the old value.

If you restart the daemon, it should start with a close old drift value and quickly converge to the newer slightly different value. If you reboot, expect it to converge to a new/different drift value and that may take a while depending on how different the basic calibration factors are.

This section deals specifically with ntpd. It discusses algorithms used by the ntpd suite to measure and correct the system time. It is not directly applicable to chronyd, although some design considerations may be similar.

You can’t optimize what you can’t visualize. The easiest way to visualize ntpd performance is with ntpviz from [NTPSEC.ORG]. Once you are regularly graphing your server performance it is much easier to see the results of changes.

The easiest way to determine the offset with chronyd is probably to configure the source whose offset should be measured with the noselect option and a long poll, let chronyd run for at least 4 hours and observe the offset reported in the chronyc sourcestats output. If the offset is unstable, wait longer. For example:

SHM 0 configured as: refclock SHM 0 poll 8 filter 1000 noselect

In this case (Garmin 18x) the offset specified in the config for the SHM 0 source should be around 0.495.

By now if you have a good serial PPS signal your local clock should have jitter on the order of 1 uSec. You do not want the hassle of placing a GPS receiver on each of your local computers. So you install chrony or ntp on your other hosts and configure them to use your NTP PPS server as their local server.

With your best setup on a lightly loaded GigE network you find that your NTP clients have jitter on the order of 150 uSec, or 150 times worse than your master. Bummer, you want to do much better, so you look to the Precision Time Protocol [PTP] for help. PTP is also known as IEEE 1588.

With PTP you can easily synchronize NTP hosts to 5 uSec with some generic NIC hardware and newer Linux kernels. Some of the Ethernet drivers have been modified to time stamp network packets when sending and receiving. This is done with the new SO_TIMESTAMPING socket option. No hardware support is required.

A more recent addition is PTP Hardware Clock (PHC) support. This requires hardware support in the NIC.

Software timestamping is more mature, available on more NICs, and almost as accurate as hardware timestamping. Try it first. This HOWTO will build on those results.

One final wrinkle before proceeding with PTP. Ethernet ports have something called [EEE] (IEEE 802.3az). Percentage wise EEE can save 50% of the Ethernet energy needs. Sadly this is 50% of an already small energy usage. Only important in large data centers. EEE can be very disruptive to timekeeping. Up to almost 1 Sec of errors in offset, wander and jitter. To see if you have EEE enabled, and then turn it off:

[NTP-FAQ] has good advice on things to be sure you have done — and are ready to do — before becoming a public server. One detail it doesn’t mention is that you’ll need to un-firewall UDP port 123. The NTP protocol does not use TCP, so no need to unblock TCP port 123.

If and when you are ready to go public, see [JOIN-POOL].

Beat Bolli <bbolli@ewanet.ch> wrote much of the section on NTP performance tuning. Hal Murray <hmurray@megapathdsl.net> wrote much of the section on NTP working and performance details. Sanjeev Gupta <ghane0@gmail.com> assisted with editing. Shawn Kohlsmith <skohlsmith@gmail.com> tweaked the Bibliography. Jaap Winius <jwinius@rjsystems.nl> cleaned up some terminology.

The loopstats-based tuning procedure for ntpd was drafted by Sanjeev Gupta <ghane0@gmail.com>, based on discussions on the GPSD list [GPSD-LIST] in Oct and Nov 2013. Code examples are based on work by Andy Walls <andy@silverblocksystems.net>. A copy of the original email can be found at [ANDY-POST]. A thorough review was contributed by Jaap Winius <jwinius@rjsystems.nl>.

This file is Copyright 2013 by the GPSD project
SPDX-License-Identifier: BSD-2-clause