PyrOSM: working with Open Street Map data

Efficient geospatial manipulations for OSM map data

If you’ve worked with OSM data before, you know it’s not the easiest to extract. OSM data can be huge, and finding performant solutions for what you want to analyze is often a challenge. PyrOSM is a package that makes the process of reading in and working with OSM data much more efficient. How? Well, PyrOSM is build on Cython (C Python) and it uses faster libraries for deserializing OSM data as well as smaller optimizations like numpy arrays which allows it to process data fast. Especially if you’ve used OSMnx before (for very similar usecases), you know that large datasets take a very long time to load into memory, which is where PyrOSM can help you work with them. Let’s get into what this library can do!

🌎 PBF Data

Let’s talk a bit about the specific file format that OSM data comes in. PBF stands for “Protocolbuffer Binary Format” and it is very efficient for working with OSM data is stored. OSM data is organized in “fileblocks”, which are groups of data that can be independently encoded or decoded. Fileblocks contain PrimitiveGroups, which in turn include thousands of OSM entities, like nodes, ways and relations.

The data can be scaled according to the user’s desired level of granularity. For instance, the current OSM database’s resolution is around ~1 cm. In fact, if you wanted, you could download the entirety of Open Street Maps data into one file, known as Planet (around 1000 Gb of data)!

👩‍💻 PyrOSM Basics: reading in datasets

PyrOSM is a package that reads in Open Street Map’s PBF data based on two main data distributors: Geofabrik (world and country-level data) and BBBike (city-level data). The package allows the user to access many types of features:

• Buildings, POIs (points of interest), Land Use
• Street Networks
• Custom Filters
• exporting as networks
• and more!

There are 235 cities across the world currently supported by BBBike, and you can get access to the full list easily by calling the “sources.cities.available” method. Getting started is easy enough, you simply initialize an OSM reader object and load in the data you want:

website that lets you make rectangles and then copy the coordinates. Here’s how to bound the area around UC Berkeley’s campus and get its walking network:

was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.