from single track …..
Over the summer there was an interesting discussion in the comments section of this article about the accuracy of dedicated GPS units from companies like Garmin compared to the GPS data collected via smartphones. I’ve personally been biking with a GPS since 2001 (13 years!) and my take was that neither is any more or less accurate than the other. Still, I was curious: was there any difference in accuracy between various types of GPS devices?
To find out, ckdake and I took 10 GPS devices–including 2 wristwatches, 2 bar-mounted units, 3 smartphones, 1 GPS-equipped helmet camera, 1 handheld device, and even a tablet–to a local quarter-mile track to see how each performed. We did our best to mount each unit according to its normal configuration; that is, wristwatches on a wrist, bar-mounted units on the bars, etc. All were rigged to a single rider (me!) and the test started and stopped at the same point on the track after riding 10 laps (2.5 miles). I stuck to the middle of lane 1 (thanks for the velodrome tip Chris!) and each GPS was started (but not moved) one at a time. We didn’t test timing on the devices since each unit is designed to sync time with the GPS satellites themselves, which meant we wouldn’t need to start and stop all devices at the same time.
Here are the results, ordered by distance accuracy.
Looking at the animated graphic at the top of this article, you can get a sense for another measure of accuracy–basically the tightness and “correctness” of the ride data. The actual route hugged the inside lane for each lap, though you can see some of the data veers toward the inside of the field and the edges in some cases (click here for an interactive plot). Subjectively, here’s how I would rank the accuracy based on the route plot:
Top 5: Asus Nexus Tablet (Strava), Nokia Lumia (GPS Logger), Garmin Fenix2, Magellan Cyclo505, and Garmin Forerunner 405.
Bottom 5: Garmin GPSMap60CSx, iPhone 5 (Strava), Garmin Edge 500, iPhone 5 (Garmin Fit), and Garmin VIRB Elite.
Interesting to note that the most accurate in terms of distance are not necessarily the most accurate at plotting route data. Keep this in mind when using GPS to map trails.
Accuracy
Overall, the error rate is pretty low–around 3% or less for all but one device. Still, only 4 out of 10 were within 1% of the actual distance. On average (if we throw out the Garmin 60CSx) each device is off by about 1.4%, which is probably a good number to assume whenever analyzing your own data.
Error Direction
Surprisingly, all but one of the devices over-reported the distance, which either means our track is slightly longer than we assumed or something else is going on. In my own experience I’ve noticed most GPS units report distance on the high side (for example, clocking 104 miles in a 100-mile race), and perhaps it’s by design to make consumers feel faster/stronger than they really are. Product engineers know the device isn’t completely accurate so why not just estimate everything on the high side to keep riders happy? 
Another more serious possibility is that distance was added to each track during the time it took us to start and stop all the devices during our test (you can see this in the lower right corner on some of the plots above). Basically if you take pretty much any GPS, hit start, and set it on the ground, after a few minutes you’ll find that your GPS has logged several feet of distance without moving. How is this possible?
GPS units never know exactly where you are located–they’re only accurate to within a few feet. So each time a GPS checks with the satellite, it’s calculating a slightly different position, even when you’re not moving.
Driving Factors
Perhaps one of the biggest misconceptions about GPS accuracy is that it’s all about the chipset (like the SiRFstarIII) in a particular device. While it’s true that some GPS chips are more powerful and/or sensitive than others, there are many, many factors that can affect distance calculations.
But let’s stick with the idea that chipsets affect distance accuracy for a moment. Two of the top four GPS units in our distance accuracy test are regular old smartphones that determine location not just with GPS but also using wi-fi and cell tower signals to triangulate a position. And the Magellan Cyclo505–#4 above–is also wi-fi enabled, though we don’t know if it’s actually using wi-fi to improve location accuracy. Still, having more sensors available to calculate location isn’t a guarantee of accuracy–the Garmin Fit app on an iPhone 5 ended up middle of the pack in our distance test, and the Nokia Lumia smartphone ranked 8 out of 10.
So what the heck happened to the Garmin GPSMap60CSx–it was off by nearly 10%?! There’s another factor at play here, and it’s known as polling frequency. Basically, GPS units check in with the satellite on a regular basis to get the user’s position and save that information, along with the time, to record your ride. Many apps and GPS units allow you to set the frequency yourself. For example, if you record your position once per second, you’ll get better data than if you record only once per minute. The tradeoff is more frequent polling will drain your battery faster and fill up your memory quicker.
In the case of our track test, the GPSMap60CSx was set to record much less frequently than the other units, resulting in “cut off” corners that significantly shortened the distance (remember, a straight line is always the shortest distance between two points). Fortunately this is an easy fix and has nothing to do with the accuracy of the unit itself.
This is what more frequent polling looks like (specifically, 1 point per second):
Some Garmin units (and perhaps units from other GPS manufacturers as well) have a “smart polling” option that doesn’t have a set polling frequency. Instead, the GPS records a point only if it’s not in a straight line with the point before it. A couple of the units we tested used this method which you can see below. In this test smart polling didn’t seem to affect accuracy but for a less-regular course (say, a twisting MTB trail) the GPS may have trouble keeping up with the rider’s constantly-changing vector.
Software Manipulation
One final factor in our GPS distance accuracy test is the use of post-ride software processing. Basically if the GPS data is uploaded to a website (Strava, Garmin Connect, MapMyRide, etc.), chances are that the data will be manipulated. This is particularly true for elevation data (which we’ll discuss in a follow-up post), but I suspect even distances are being tweaked behind the scenes for some of the devices we tested.
In particular, it’s either incredibly lucky that the Strava iPhone app got within 0.1% of our target distance or that Strava is guessing (correctly) that our “workout” occurred on a quarter-mile track and is adjusting distances accordingly. Looking at the trace from the Strava iPhone app, it’s easily the messiest and least “tight” trace which, again, suggests it was either lucky or is making distance adjustments behind the scenes. The fact that the second-most accurate distance tracking also came from the Strava app but running on a low-powered Android tablet lends some credibility to the latter suggestion*.
Conclusions
This is by no means a definitive GPS distance accuracy test. In fact, if we conduct this test again (which we probably will at some point), my hypothesis is the ranking will be completely different. And aside from the track test, a test in an area with zero cell phone or wi-fi signals over a known distance (not a quarter-mile track) could help isolate the additional sensor effect and the software manipulation issues. Bottom line: if you’re choosing a GPS device or app for clocking your rides, don’t focus too much on accuracy… they’re all pretty good, but none are bulletproof.
* Being curious, I tested the theory that Strava is recognizing track workouts–and modifying them–by uploading raw data from one of the other devices used during the test. The result: no change in the distance. Still not definitive, but that theory is looking shaky.
Kite*Surf*Bike*Rambling 