OSM Derivative Database - Derivation Methodology


At AllTrails, we have derived trail segments from OpenStreetMap data and have stored them in a separate derived database, used in the creation of our AllTrails Map Layer TileSet and our custom map creation tool.


This document makes publicly available the method of making the alterations to the original OSM "Database," as required of the ODBL license in section 4.6.b.


Unlike the OSM database, which depends on nodes and ways, where ways represent entire roads/trails and nodes represent the ends of these roads/trails, we depend on the idea of segments (i.e. individual segments that compose a way and start/end at the intersections of ways). The OSM database previously included segments, but these were removed in October 2007 to simplify the data model (according to the following documentation: https://wiki.openstreetmap.org/wiki/Segment).


The algorithm to create our derivative database is as follows:


1. Use the Overpass API to download OSM way data in 2x2 degree tiles, between -58 and +72 latitude:

"http://www.overpass-api.de/api/xapi?way[highway=path|track|footway|steps|bridleway|cycleway][bbox=#{lng},#{lat},#{lng+TILE_DIM},#{lat+TILE_DIM}]"

Data will be download in .osm format.


2. Convert each 2x2 tile from .osm to .geojson format using OsmToGeojson for easier parsing ability.

3. Iterate over the ways contained in each tile, storing way information into a hash keyed by way id (necessary because the same way may appear in multiple 2x2 tiles)

5. Iterate over each way (A) found in the hash, for each way (A), iterate over the points of all other ways (B) and compare points to the original way (A), until a match has been found. At the point where these ways match, store an entry into another hash, also keyed by way id, but with a value representing an array of all intersections of this way (A) with all other ways (B).

6. Iterate over each way (A) again, this time, segmenting each way by each intersection found and stored in the hash created in the last step. 

7. Calculate total distance for each way segment by calculating the cartesian distance between each point of the way segment and then summing the values.

8. Calculate the bounding box of the way segment by find the minimum and maximum latitudes and longitudes of all points in the way segment and then forming each corner of the bounding box as each unique combination of the four min/maxes. 

9. Output each intersection and/or segment to Geojson or other format, storing unique way segment data as needed, such as:

Intersections - id (arbitrary, but unique), latitude, longitude

Way Segments - original OSM way id, points_data, intersection_start_id, intersection_end_id, distance_total, bounding_box, private_access (access=private, access=no), bicycle_accessible (bicycle=yes)