Driving Log Version 2.0: Canonical KML File and Import Script

I’ve been mapping my driving since 2010. Most of this driving data came from Garmin automotive GPS units I’ve owned over the last eleven years, which all save their latitude and longitude up to once per second¹ while they’re turned on, so I have a lot of data to deal with.

Previously, I just kept my driving log in a lot of Google Earth KMZ files—one for every trip I’d driven on, plus a few large “local driving” files that covered various time frames. Some of these files had all driving merged into a single track, and some of the files kept individual drives separate. All my data was there, but it wasn’t easy to work with—especially once I started making more involved maps in QGIS, and needed finer control of the data than “just drag every file into Google Earth to see everything at once.”

It was also a bit of a pain, since every time I downloaded data from my GPS I had to convert it to KML. GPS Visualizer is great, but manually uploading, converting, downloading, and styling the data every time was still a time-consuming manual process.

I started to think about if there was a better way for me to store my driving data, and whether I could automate my driving log imports.

What I Want to Do with the Data

First and foremost, I use my driving log to see where I’ve been. I like to explore new places, so having a map of where I have been often shows gaps in my travel that I might otherwise have missed. Ideally, I should be able to use my smartphone to see a map of everywhere I’ve driven around wherever I am, so no matter where I’m traveling I can find new places.

I also publish an annual year in travel summary, which now includes maps of all my driving for the year. So I need to be able to filter all my driving to only show data for a specific year.

File Type Considerations

My Garmin GPS units store driving data in GPX format. GPX is nice since nearly any mapping software can use it, but it’s also very large; every point has a lot of XML data associated with it, and when you’re recording a point as often as once per second, it gets unwieldy pretty fast.

KML is also an XML-based format, but it records a driving track as a simple string of coordinates, so it’s a lot smaller. (See Generating GPX and KML Maps with Ruby on Rails for examples of the same data in GPX and KML.) KML files can also be zipped and stored as KMZ to make them even smaller. It’s Google’s native map data format, so it can be opened in Earth without any conversion. And it supports adding a timestamp to every track, which would allow me to filter by year.

GeoPackage is a bit different—it’s a way of storing geospatial data in an SQLite database, which means that I would have a single file that I could run SQL queries on (which would make filtering by date really easy). However, Google Earth doesn’t support it natively (I’d have to export KML data every time I updated the GeoPackage file). It also stores the track data in a binary format rather than human-readable text—not necessarily a deal-breaker, but it adds an extra conversion step.

Given all of that, I ultimately decided to store all of my driving data in a single KML file.

KML File Layout

My KML file is a simple collection of <Folder> elements under the <Document>. Each one contains a single <LineString> that represents a single driving track segment—a collection of consecutive waypoints for a single, unbroken path from an origin to a destination.

The <LineString> has a <coordinates> element, which contains a string of space-separated longitude,latitude,altitude triples:

<coordinates>
  -81.697262,41.504192,175.58 -81.69745,41.50393,175.1
  -81.698268,41.503463,175.1 -81.699067,41.503422,181.35
  -81.699243,41.503153,181.35 -81.699314,41.503045,184.71
  -81.69934,41.503006,185.67 -81.699367,41.502966,186.15
  …
</coordinates>

The <Folder> also has a <TimeStamp>, with a <when> tag that contains the UTC timestamp for the start time of the track. QGIS and Google Earth can both use this attribute to filter which tracks are visible based on a date range.

I also used a string representation of the timestamp as the <name> of the <Folder>.

The <Style> defines what every track looks like (I selected a 4px red line), and each <styleUrl> references this style so I don’t have to waste space defining the style for thousands of tracks.

This all results in a single file with thousands of tracks at the root level. My unzipped Driving.kml file (with data from March 2010 through February 2021) is about 41 MB, and the zipped Driving.kmz reduces these eleven years of data to about 11 MB. Google Earth is more than capable of handling a file of this size on any computer I own, including the Google Earth app on iOS. If the size ever does get too large, though, I could conceivably split my KML data into a separate file for each decade.

Importing GPX Data

Google Earth can import GPX files, so it would certainly be possible to manually update my KML file by importing my Garmin’s most recent GPX and then copy/pasting track segments into my Driving.kml file. Although this would be faster than the GPS Visualizer and multiple files approach I discussed in the introduction, it would still be unwieldy, particularly since I would still have to apply styles to every pasted track segment.

Instead, I wrote a Python script which would read tracks from my existing Driving.kml file and a supplied GPX file, merge them (removing any duplicate tracks), and save them as both Driving.kml (overwriting the existing file) as well as a zipped Driving.kmz. Effectively, Driving.kml is my canonical driving data source, while day-to-day I can work with the smaller Driving.kmz.

Using KMZ on Mobile

I use OneDrive for cloud file synchronization, and I keep my Driving.kmz file under my OneDrive folder. This makes it easy to open on any of my computers, but it’s also pretty easy to open it on my iPhone or iPad.

Both the Google Earth app and OneDrive app must be installed on iOS in order to open a KMZ file from OneDrive. (I’d assume this would work with other cloud storage providers as well, but I haven’t personally tested them.)

From the hamburger menu (three horizontal bars) on the Google Earth app, I can select Projects > Open > Import KML File. From there, there’s a list of locations which includes OneDrive; selecting it, I can browse through my OneDrive folders and open Driving.kmz.

Generating Yearly Driving Maps

I use QGIS to create my year in travel driving maps. Within QGIS, I can import Driving.kml as a vector data source. This shows my entire driving log on the map, and I can create a filter on the timestamp attribute to show my driving for a particular year:

"timestamp" >= '2020-01-01' AND "timestamp" <= '2020-12-31'