While working on my various projects, I’ve dealt with various types of maps.
My Flight Historian plots flight data using Great Circle Mapper. These maps are simple to generate from my flight data (I just have to pass it a plain text collection of airport codes) and easy to embed in my website. However, because they are static images, they can’t be easily panned, zoomed, or otherwise manipulated in the way that modern map websites and apps can.
A sample Great Circle Mapper map of my flights in 2018.
Map generated by Paul Bogard using the Great Circle Mapper - copyright © Karl L. Swartz
On the other hand, my driving maps require too much detail for a single image, so I create them in Google Earth, which lets me manipulate the view as much as I need to. The driving data is a bit more complicated than my flight log data; while my flight log represents the abstract shortest distance straight line between two airports (and thus only requires specifying the airport at each end), a single drive can involve tens of thousands of coordinates that can be joined together, connect-the-dots style, to show the actual driving route taken.
A sample driving data map using Google Earth.
Fortunately, all those coordinates are automatically generated and saved by my car’s GPS navigation unit in a file format called GPX (GPS Exchange Format), which is an XML-based file format which contains (among other things) latitude/longitudes sampled once per second (in the case of my particular GPS). For example, here’s a sample from a GPX file from a recent trip:
…
<trk>
<trkseg>
…
<trkpt lat="37.655198" lon="-97.427427">
<ele>399.570</ele>
<time>2019-02-25T02:29:41Z</time>
</trkpt>
<trkpt lat="37.655257" lon="-97.427516">
<ele>399.570</ele>
<time>2019-02-25T02:29:42Z</time>
</trkpt>
<trkpt lat="37.655311" lon="-97.427624">
<ele>399.570</ele>
<time>2019-02-25T02:29:43Z</time>
</trkpt>
<trkpt lat="37.655367" lon="-97.427735">
<ele>399.570</ele>
<time>2019-02-25T02:29:44Z</time>
</trkpt>
<trkpt lat="37.655425" lon="-97.427851">
<ele>399.080</ele>
<time>2019-02-25T02:29:45Z</time>
</trkpt>
…
</trkseg>
</trk>
…
Google Earth doesn’t use GPX format (though it can import it); instead, it uses a format called KML (Keyhole Markup Language, from back when Google Earth was Keyhole EarthViewer). KML is also an XML-based format, so conceptually it’s similarly a collection of coordinates that can all be joined together, with its own slightly different style. Here’s the same data from the above GPX file, shown in KML format:
…
<MultiGeometry>
<LineString>
<coordinates>
…
-97.427427,37.655198,399.57 -97.427516,37.655257,399.57 -97.427624,
37.655311,399.57 -97.427735,37.655367,399.57 -97.427851,37.655425,
399.08
…
</coordinates>
</LineString>
</MultiGeometry>
…
(Note that KML reverses the order of longitude and latitude.)
But while GPX and KML can be used to represent complicated route shapes, they don’t have to be. These formats are both just as capable of taking a pair of points on the globe and drawing the shortest line between them. With that in mind, I decided to try to have Flight Historian automatically generate KML and GPX versions of my flight map, which would let me show my flight routes in Google Earth and Google Maps.
Table of Contents
Overall Strategy
I already had a Map class for generating Great Circle Mapper maps, which had several more specific child classes (FlightsMap,
SingleFlightMap, HighlightedRoutesMap, AirportsMap, andAirportFrequencyMap). Each of these child classes would accept certain combinations of airports and flight routes when a new instance was created, and would distill them down to arrays of airport IDs (a separate array for each map airport display style, such as airports_normal or airports_highlighted, in the format [1,2,3]) and arrays of pairs of airport IDs (a separate array for each map route display style, such as routes_normal or routes_highlighted, in the format [[1,2],[3,4],[5,6]]).
The parent Map class had a Map.draw method which would use these arrays to generate the HTML necessary to embed a Great Circle Mapper map image.
Since Map already did the work of converting all the various map styles into airport and route arrays, I decided that I could make Map more generic so it could also create GPX and KML maps. I renamed the Map.draw method to Map.gcmap (which still returns HTML), and created new Map.gpx and Map.kml methods, both of which return XML.
Creating Map.gpx
As far as Ruby on Rails is concerned, creating XML isn’t substantially different from creating HTML. Effectively, you can use the same technique you’d use to return inline HTML in a helper method.
Of course, to have a Class act like a Helper, I needed to explicitly include some ActionView modules (which I’d already done for the gcmap HTML):
class Map
include ActionView::Helpers
include ActionView::Context
GPX Structure
I needed to generate XML tags in accordance with the GPX schema. The basic structure of XML I needed to generate was as follows:
<?xml version="1.0" encoding="UTF-8" ?>
<gpx xmlns="http://www.topografix.com/GPX/1/1" version="1.1">
<metadata>
<name>Flights</name>
<desc>Map of flight routes, created by Paul Bogard’s Flight Historian</desc>
<author>
<name>Paul Bogard’s Flight Historian</name>
<link href="https://www.flighthistorian.com">
<text>Paul Bogard’s Flight Historian</text>
</link>
</author>
</metadata>
<wpt lat="33.6366996" lon="-84.427864">
<name>ATL / KATL</name>
<description>Atlanta</description>
</wpt>
<!– Repeat wpt tag for every airport –>
<rte>
<name>ATL–BOS</name>
<desc>Atlanta – Boston</desc>
<link href="https://www.flighthistorian.com/routes/ATL/BOS" />
<rtept lat="33.6366996" lon="-84.427864">
<name>ATL / KATL</name>
<description>Atlanta</description>
</rtept>
<rtept lat="42.3629444" lon="-71.0063889">
<name>BOS / KBOS</name>
<description>Boston</description>
</rtept>
</rte>
<!– Repeat rte tag for each route –>
</gpx>
Metadata
The <metadata> section was pretty simple, as it’s just a static string. (Technically, it needed to be an ActiveSupport::SafeBuffer to get the raw XML, since Rails escapes HTML/XML it finds in strings. A SafeBuffer is what you get when you run .html_safe on a string, though it’s better to use content_tag since .html_safe can be pretty dangerous when used on anything non-static. However, I do use html_safe on the XML prolog, since that never changes, and content_tag can’t handle that syntax.)
def gpx
output = %Q(<?xml version="1.0" encoding="UTF-8" ?>).html_safe
output += content_tag(
:gpx, xmlns: "http://www.topografix.com/GPX/1/", version: "1.1"
) do
concat (content_tag(:metadata) do
concat content_tag(:name, map_name)
concat content_tag(:desc, map_description)
concat (content_tag(:author) do
concat content_tag(:name, "Paul Bogard’s Flight Historian")
concat content_tag(:link, content_tag(
:text, "Paul Bogard’s Flight Historian"
), href: "https://www.flighthistorian.com")
end)
end)
# (airport code here)
# (route code here)
end
return output
end
Waypoints (Airports)
The <wpt> tags required that I loop over the airport arrays, generating XML tags for each. Specifically, the way I wrote Map, the airport arrays are arrays of airport IDs from my Airports database table. The Map class already had a class variable @airport_details which is a hash of all airports used by the map, in the following format:
@airport_details[9] = {
iata: "ATL",
icao: "KATL",
latitude: 33.63670,
longitude: -84.42786,
city: "Atlanta",
country: "United States"
}
I created a Map.gpx_airports method which would accept an array of airport IDs and loop through them, and a gpx_airport method which would generate the XML for a specific waypoint:
def gpx_airports(airports)
airports = airports.sort_by{|a| @airport_details[a][:iata]}
return safe_join(airports.map{|a| gpx_airport(a, :wpt)})
end
def gpx_airport(airport_id, wpt_type)
detail = @airport_details[airport_id]
return content_tag(
wpt_type, lat: detail[:latitude], lon: detail[:longitude]
) do
concat content_tag(:name, detail[:iata] + " / " + detail[:icao])
concat content_tag(:description, detail[:city])
end
end
Note that the gpx_airport method has a wpt_type parameter. This is because while GPX waypoints are contained within a <wpt> tag, GPX route tags also contain a collection of routepoints, which are functionally identical to waypoints but are contained within a <rtept> tag. Thus, I wrote the function so it could be used for either, and the wpt_type parameter specified which type I wanted to use.
Routes
Similarly, I wrote a Map.gpx_routes method to loop over an array of pairs of airport IDs, and a Map.gpx_route method to generate the appropriate XML for a specific route:
def gpx_routes(routes)
return nil unless routes.any?
routes = routes.map{|r| r.sort_by{|x| @airport_details[x][:iata]}}
.uniq
.sort_by{|y| [@airport_details[y[0]][:iata], @airport_details[y[1]][:iata]]
}
return safe_join(routes.map{|r| gpx_route(r)})
end
def gpx_route(airport_pair)
detail = airport_pair.map{|a| @airport_details[a]}
return content_tag(:rte) do
concat content_tag(:name, detail.map{|a| a[:iata]}.join("–"))
concat content_tag(:desc, detail.map{|a| a[:city]}.join(" – "))
concat content_tag(:link, nil,
href: "https://www.flighthistorian.com/routes/" +
"#{detail[0][:iata]}–#{detail[1][:iata]}"
)
concat safe_join(airport_pair.map{|a| gpx_airport(a, :rtept)})
end
end
Completing the Maps.gpx Method
For GPX (unlike for Maps.gcmap), I wasn’t planning to format different categories of airport or routes differently, since the GPX format doesn’t really have formatting without using extensions. Thus, I could just loop over the keys of the entire @airport_details hash, rather than bothering with airports_normal, airports_highlighted, etc. Routes didn’t have quite a simple way to merge them all, so I just ended up doing a set union (|) between all of my routes arrays.
def gpx
output = %Q(<?xml version="1.0" encoding="UTF-8" ?>).html_safe
output += content_tag(
:gpx, xmlns: "http://www.topografix.com/GPX/1/1", version: "1.1"
) do
concat (content_tag(:metadata) do
concat content_tag(:name, map_name)
concat content_tag(:desc, map_description)
concat (content_tag(:author) do
concat content_tag(:name, "Paul Bogard’s Flight Historian")
concat content_tag(:link, content_tag(
:text, "Paul Bogard’s Flight Historian"
), href: "https://www.flighthistorian.com")
end)
end)
concat gpx_airports(used_airports)
concat gpx_routes(
routes_normal | routes_out_of_region |
routes_highlighted | routes_unhighlighted
)
end
return output
end
And with that, calling gpx on a Map instance will return a SafeBuffer containing GPX-formatted XML.
Creating Map.kml
Conceptually, creating KML was pretty similar to creating GPX—create an XML structure, and loop through airports and routes to create XML elements. However, KML does have somewhat more support for styling, so while I chose to maintain a single style for all airports, I did end up distinguishing between the various types of routes.
KML Structure
The KML schema dictates a document like this:
<?xml version="1.0" encoding="UTF-8" ?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<Document>
<name>Flights</name>
<description>
Map of flight routes, created by Paul Bogard’s Flight Historian
</description>
<Style id="airportMarker">
<Icon>
<href>
http://maps.google.com/mapfiles/kml/shapes/placemark_circle.png
</href>
</Icon>
</Style>
<Style id="flightPath">
<LineStyle>
<color>ff0000ff</color>
<width>2</width>
</LineStyle>
</Style>
<Folder>
<name>Airports</name>
<Placemark>
<name>ATL / KATL</name>
<description>Atlanta</description>
<styleUrl>#airportMarker</styleUrl>
<Point>
<coordinates>-84.427864,33.6366996,0</coordinates>
</Point>
</Placemark>
<!– Repeat Placemark tag for every airport –>
</Folder>
<Folder>
<name>Routes</name>
<Placemark>
<name>ATL–BOS</name>
<styleUrl>#flightPath</styleUrl>
<LineString>
<tessellate>1</tessellate>
<coordinates>
-84.427864,33.6366996,0 -71.0063889,42.3629444,0
</coordinates>
</LineString>
</Placemark>
<!– Repeat Placemark tag for every route –>
</Folder>
</Document>
</kml>
Metadata and Style
Unlike GPX, KML doesn’t have a specific <metadata> tag. Instead, metadata (like name and description) are kept in the main <Document> element.
KML supports named styles within a <Style> tag, also kept in <Document>. I defined a specific icon for airports, and defined a red line with a width of 2 for route lines. (Note that the colors are defined as a hexadecimal color with alpha, but instead of RRGGBBAA, it’s AABBGGRR.)
I created a Map.kml_styles method to keep these in one place:
def kml_styles
output = content_tag(:Style, id: "airportMarker") do
content_tag(:Icon) do
content_tag(:href,
"http://maps.google.com/mapfiles/kml/shapes/placemark_circle.png"
)
end
end
output += content_tag(:Style, id: "flightPath") do
content_tag(:LineStyle) do
concat content_tag(:color, "ff0000ff")
concat content_tag(:width, "2")
end
end
end
Points (Airports)
The basic map element for KML is the <Placemark>. While these can be put directly in the <Document> element, they can also be placed in a <Folder> (if you’ve seen folders in the left sidebar on Google Earth, that’s what these are). I decided I’d create a folder for airports, and (in the next section) a folder for routes. The folder has a single <name>, and a <Placemark> for each airport.
In order to use the icon style I defined above, I needed to include the name of the style with <styleUrl>#airportMarker</styleUrl>.
The <Placemark> contains a <Point>, which contains <coordinates>. Note that the coordinates are in longitude,latitude,altitude order, NOT latitude,longitude,altitude. Since I don’t care about altitude for this map, I always set it to zero.
I created a Map.kml_airports method which would loop through an array of airport IDs, and a Map.kml_airport method which would generate the XML for a specific point:
def kml_airports(airports)
airports = airports.sort_by{|a| @airport_details[a][:iata]}
return content_tag(:Folder) do
concat content_tag(:name, "Airports")
concat safe_join(airports.map{|a| kml_airport(a)})
end
end
def kml_airport(airport_id)
detail = @airport_details[airport_id]
return content_tag(:Placemark) do
concat content_tag(:name, detail[:iata] + " / " + detail[:icao])
concat content_tag(:description, detail[:city])
concat content_tag(:styleUrl, "#airportMarker")
concat content_tag(:Point, content_tag(:coordinates,
"#{detail[:longitude]},#{detail[:latitude]},0"
))
end
end
LineStrings (Routes)
Similarly, I created a <Folder> for routes. I still needed to use <Placemark> tags to create lines, but within them are <LineString> tags.
For long-distance lines (such as flight routes), the <tessellate>1</tessellate> option needs to be included to prevent the line from clipping below the terrain.
The <coordinates> element uses the same longitude,latitude,altitude format as points did above, but since it’s a line, it needs multiple coordinates, which are all kept within the same tag, space-separated.
Not surprisingly, I wrote Map.kml_routes and Map.kml_route methods to create a collection of routes (from an array of pairs of airport IDs) and an individual route, respectively:
def kml_routes(routes, name)
return nil unless routes.any?
routes = routes.map{|r| r.sort_by{|x| @airport_details[x][:iata]}}
.uniq
.sort_by{|y| [@airport_details[y[0]][:iata], @airport_details[y[1]][:iata]]
}
return content_tag(:Folder) do
concat content_tag(:name, name)
concat safe_join(routes.map{|r| kml_route(r)})
end
end
def kml_route(airport_pair)
detail = airport_pair.map{|a| @airport_details[a]}
return content_tag(:Placemark) do
concat content_tag(:name, detail.map{|a| a[:iata]}.join("–"))
concat content_tag(:styleUrl, "#flightPath")
concat (content_tag(:LineString) do
concat content_tag(:tessellate, "1")
concat content_tag(:coordinates, detail.map{|a|
"#{a[:longitude]},#{a[:latitude]},0"
}.join(" "))
end)
end
end
Completing the Maps.kml Method
Putting it all together looks pretty similar to what I did for GPX, though I did end up separating my different route types in different folders:
def kml
@airport_details ||= airport_details
used_airports = @airport_details.keys
output = %Q(<?xml version="1.0" encoding="UTF-8" ?>).html_safe
output += content_tag(:kml, xmlns: "http://www.opengis.net/kml/2.2") do
content_tag(:Document) do
concat content_tag(:name, map_name)
concat content_tag(:description, map_description)
concat kml_styles
concat kml_airports(used_airports)
concat kml_routes(routes_normal | routes_out_of_region, "Routes")
concat kml_routes(routes_highlighted, "Highlighted Routes")
concat kml_routes(routes_unhighlighted, "Unhighlighted Routes")
end
end
return output
end
Serving the XML
I wanted to make a GPX and KML map of all of my flights downloadable, which meant I needed to render them effectively as XML views (as opposed to the usual HTML or HTML ERB views of a Ruby on Rails application).
Fortunately, rendering an XML view is not much different from rendering an HTML view.
Controller Actions
First, I needed to create a controller action for each of the XML maps. Since I was creating flight maps, I decided to place them in flights_controller.rb:
def show_flight_gpx
flights = flyer.flights(current_user)
render xml: FlightsMap.new(flights).gpx
end
def show_flight_kml
flights = flyer.flights(current_user)
render xml: FlightsMap.new(flights).kml
end
flyer.flights(current_user) is a method I already had to return all flights that a user of the site has permission to see.
Normally, Rails would automatically look for a view matching the name of the controller action. However, I overrode that behavior here and explicitly told Rails to render an XML document instead by using the render xml: syntax. (This also meant I didn’t need to add anything to the /app/views/flights folder!)
As discussed above in Overall Strategy, FlightsMap is a child class to Map, and so instances of FlightsMap respond to .gpx and .kml by returning XML.
Effectively, each controller action gets all flights the user has permission to, creates a map, gets the GPX or KML XML data from that map, and renders it as XML.
Routing
While Rails doesn’t require a file extension on routes, I was ultimately trying to generate KML and GPX files, so I went ahead and gave the routes .kml and .gpx extensions so they had a logical filename when downloaded.
get "/flights/flights_map.gpx" => "flights#show_flight_gpx", as: :show_flight_gpx
get "/flights/flights_map.kml" => "flights#show_flight_kml", as: :show_flight_kml
Sample Output
Conclusion
Conceptually, generating GPX and KML (or any kind of XML) aren’t too different from creating a standard HTML/ERB view in Rails—either one is just a matter of generating the right tags.
I still want to spend some time playing around with styles in KML in particular. I’d also like to add more metadata to each waypoint/point and route/linestring (such as number of times visited or flown, distance, etc.).
But for now, I can export my flight maps into other mapping applications!