Moving to GPSD-NG: a Guide for Client Developers

GPSD has moved to a new request/response protocol. This move has been forced upon us because the old one ran out of namespace. It was designed with case-insensitive single-character command/response codes. 25 of 26 ASCII alphabetics were already in use by 2006, and there have been functional challenges accumulating over the last three years that will require several more request/response codes - things like reporting AIS data, reporting raw pseudoranges, and reporting RTCM3.

Yes, we could as a desperate expedient have pressed non-alphabetic printables into service - but the result would have looked like line noise and only delayed the day of reckoning. Instead, we’ve written a new protocol that is upward-compatible with the old one in that you can mix old and new syntax.

There were other problems as well. The old command set encouraged sloppy handling of data by supporting commands that return PVT and fix-quality information in atomized partial form, without timestamps.

There was also no way to support returning more than single-line responses to a client. This was a problem for returning things like [RINEX] format, which we’d like to be able to do when a device can report pseudoranges.

We need to shed the code complexity and overhead of maintaining both protocols at once. This will happen sooner than it otherwise might have because gpsd is in part targeted for SBCs and other constrained environments where shaving a few K off the runtime image can actually matter. When it comes to keeping the codebase lean and mean, we try harder.

The old-protocol support was removed from the daemon in 2.91. Old protocol will be supported in the client-side library for somewhat longer, giving your applications a bridge period when they can speak both old and new protocol — but the client-side support for old protocol will be removed when 3.0 ships.

The 2.92 version, with the new protocol deployed, is be in Lucid Lynx, the Ubuntu LTS release of April 2010.

If you follow our transition advice now, you will be able to talk to all old-protocol and new-protocol versions of the daemon until the 3.0 shared client library is deployed, at which point your runtime will silently get smaller but you may no longer be able to use 2.x daemon versions any more.

We’re counting on binary-package dependencies to make the transition easier. When you ship a release using the new interface library, specify the GPSD package at version >= 2.90 to get the new shared library; then, when the 3.0 interface library is deployed next year, you shouldn’t have to do anything.

We’ll try to make the transition easy, but we cannot guarantee no problems. The sooner you start adapting your code, the less pain you are likely to experience. The rest of this document will explain both theory and practice, and give you specific pointers on how to fix client code.

Since 2004, the way you were supposed to be using gpsd was through one of the client libraries (in C or Python). If you have been doing it right, you have been telling gpsd to stream data at you with the 'w' command, via application code probably looked something like this:

If you have been virtuous, you need only to make four small changes to your code.

This will tell whatever version of the client library your application dynamically links to emit what it should under the hood, either old or new protocol. Unless a target system carries a version of the libgps shared library different from the gpsd version, everything should work and continue to work through future updates.

There. You’re probably done, unless you relied on some parts of struct gpsdata_t that application developers usually don’t or issued unusual configuration commands. Here are the exceptions:

You can probably ignore the rest of this document, unless either (a) you want to learn about gpsd’s new capabilities so you can use them in creative ways, or (b) you want to caper with unholy glee as you contemplate the trials awaiting the non-virtuous.

If you are non-virtuous — that is, you rolled your own client-side interface — you’ve had years of warning that this choice would fail to insulate you from protocol details and cost you pain in the future. That future is now.

In the remainder of this document we will try to help you minimize the pain. The main strategy for doing so is to use libgps (or its functional equivalents in languages other than C). Scrap your hand-rolled interface code! When you use libgps, compatibility issues become our problem to solve rather than yours.

In the past, the GPSD project has not been very good about preserving stability of the binary structure layout for struct gpsdata_t between releases. There was a reason for this — we were very focused on reducing memory footprint for SBCs and embedded devices, so we conditioned out various pieces of the strucure depending on what features were or were not compiled in.

We’re not going to do this any more. It has been pointed out to us that the friction costs of breaking shared-library compatibility are higher than we were reckoning. The new layout has no sections conditionalized; instead, we have moved a number of fields into a union. From 2.90 on, the structure layout will change rarely, only at major version bumps.

Even virtuous clients have to worry about version skew. Supposing you have used libgps and not done anything exotic, you will still have problems if the client library you linked and the instance of gpsd it speaks to are using different protocols.

The possible failure modes are pretty obvious. Transitions are difficult. We’re essentially relying on the distribution integrators to ship libgps and gpsd updates at the same time, with sane package dependencies. If that goes smoothly, applications may not even notice the changes. We can hope…​

We have deleted the two functions in the API that managed a library-internal thread hook. Here’s why:

At the same time that pressure has been building to redesign the protocol, we’ve been gaining experience in gpsd’s application domain that has made us rethink some of the assumptions behind the old one.

Since we knew we were going to have a forced compatibility break at the wire-protocol level anyway, we decided not to do things by halves. One big break — in the application model, struct gpsdata_t, and the wire protocol behind it — is better than three or four spread out over a period of time.

As a result, the new protocol is not an exact superset of the old one. It reflects a different way of carving up the behavior space in gpsd’s application domain. And the shape of struct gpsdata_t, the client-side interface structure, has changed in corresponding ways.

Accordingly, there are three things a client developer will need to understand about the new protocol. The first is theory: how its model of the gpsd application domain is different. The second is practice: how to issue new-style commands and interpret responses. The third, if you have relied on the structure in a way that now breaks your compile, is how that structure has changed.

If your code issues old-protocol commands 'A', 'D', 'E', 'M', 'P', 'T', 'U', or 'V', it is a wretched hive of scum and villainy that probably hasn’t changed since before the introduction of 'W' in 2004-2005. You are using the oldest single-shot commands and will have to rewrite your interface significantly, as the new protocol does not support equivalents. Use libgps.

If your code issues B, C, or N commands, they need to change to ?DEVICE commands. See the protocol reference for details.

The 'F' command has no equivalent in 2.90; consider teaching your client to ignore fix updates when they don’t have a specified "device" or "class" tag, respectively. In 2.91 and later versions, use the "device" option of the ?WATCH command for similar effect.

The old 'G' command does not have an equivalent. It would be possible to implement one, but we probably won’t do it unless there is actual demand (and we don’t expect any).

The old 'I' command has no equivalent. You probably issued it as part of an initialization string, hoping that a subtype string would later show up in gps_id so you could post it somewhere. In the new protocol, when a device sends back subtype information the daemon ships the client an object of class DEVICE with a device tag and subtype fields. Watch for that and process appropriately.

The old 'J' command is dead. gpsd now detects the end of the reporting cycle reliably and ships on that, buffering data during the individual reporting cycle.

The old 'K' command is replaced by ?DEVICES.

The old 'L' command is replaced by ?VERSION. Note that the daemon now ships a version response to each client on connect, so it will probably never be necessary for you to issue a ?VERSION request.

The old 'M' command has no equivalent. Mode is reported in the TPV response.

The old 'O' and 'Y' commands are gone. Use ?WATCH and sample the stream instead.

The old 'Q' command has no equivalent. DOPs are reported in the SKY response.

The 'S' command has no equivalent, because it is not well defined what value should be presented for binary protocols.

The old 'R' command has been replaced by three optional attributes in ?WATCH. Include the WATCH_RARE, WATCH_RAW and/or WATCH_NMEA masks in the argument of gps_stream(), or set a raw hook before alling gps_stream().

The old 'W' command has been replaced by ?WATCH. Call gps_stream() with whatever options you want to set.

The old 'X' command is gone. Instead, you will see an object of class DEVICE from the daemon whenever a device is opened or closed.

The old 'Z' and '$' commands, used by the developers for profiling, have equivalents, which are presently considered unstable and thus are undocumented.

gps_open() now takes a third argument and is re-entrant - it’s the old undocumented gps_open_r().

The gps_query() entry point is gone. With the new streaming-oriented wire protocol, it is extremely unwise to assume that the first transmission from the damon after a command is shipped to it will be the response to command. If you must send explicit commands to the daemon, use gps_send() and handle the response in your main event-polling loop — but beware, as using gps_send() ties your code to the GPSD wire protocol and is not recommended.

gps_poll() is renamed gps_read().

The client library’s reporting structure, struct gpsdata_t, has a new substructure (struct devconfig_t) named "dev" that groups together information about the device that shipped the last update to the client. The members of this structure replace several top-level struct gpsdata members in older versions.

Most notably, the gps_device member has been replaced by dev.path. It is valid after every response with a device tag (DEVICE, TPV, SKY, AIS, RTCM2, RTCM3).

The top-level gps_id member is replaced by dev.subtype. This data should be considered valid only when DEVICEID_SET is on in the top-level set member.

The dev members baudrate, parity, stopbits, cycle, mincycle, and driver_mode replace older top-level members. They should be considered valid only when DEVICE_SET is on in the top-level set member.

The top-level members ndevices and devicelist (used only on the client side) have been replaced by an array of struct devconfig_t structures. Data in this structure should be considered valid only when DEVICELIST_SET is on in the top-level set member. Storage for pathnames is no longer dynamically allocated, but static; to save space, it lives in a union with several other substructures.

The top-level member "satellites" has been changed to "satellites_visible". The ambiguity in that name had actually induced a bug or two.

There is a new substructure, dop, which holds the dilution-of-precision factors that were previously individual members of the gpsdata structure. Two new DOPs, xdop and ydop, are available; these express dilution of precision in longitude and latitude, respectively.

There is a gps_waiting() method analogous to the waiting() method in the Python class — a way to check if input is waiting from the daemon. It blocks but takes a timeout value.

The raw_hook member is gone.

In API version 5, the C++ library defines a single object using RAII. There are no explicit open() and close() methods; instead, you initialize the object handing it a host and server, and the connection is shut down when the object is deleted or goes out of scope.

There is a new stream() method analogous to the gps_stream() call in the C library. As in the C library, the query() method is gone, for the same reasons. The gps_send() entry point, new in version 3 of the C API, has had a corresponding Python gps-class send() method all along.

The pre-existing interface using the poll() method and self.valid is still available and should work compatibly with a daemon speaking JSON. One new feature has been added; after a VERSION response (which a JSON-speaking instance of gpsd should emit when a connection is opened) the version member of the session will be an object containing version information. However, data from the new responses (WATCH, VERSION, AIS, and TIMING in particular) will be available only through the self.data member.

The preferred way to use the new gps class is as an iterator factory, like this:

See the Client HOWTO for a more detailed example.