Creating a Google Maps Clone - Part 1 - Tiles and Styles

Time 7 minute read
Vector tiles with different styles
Vector tiles with different styles

This post is part of the Creating a Google Maps Clone series.

If you don’t care about the boring 101 stuff or you already know everything about vector tiles (why are you reading this anyway? 🤷), SKIP AHEAD!

Before we start, let’s dive into some of the basics for web mapping. Displaying geographic information in a web application usually means you need vector data of your surroundings: Points for POIs (points of interest), i.e. bus stations, shops, housenumbers etc. Lines for streets, rivers, railways etc. Polygons for buildings, parks, sport tracks etc.

There is a whole database of free geographic information available, called OpenStreetMap. You can get either the world as a whole, or download extracts at different scales, for continents, countries, states and (if they are big enough) cities from Geofabrik and other sources.

OpenStreetMap data (*.osm) is actually plain XML, but rendering these files from a server or even client is not really efficient, because it contains “because it contains far more information than can be reasonably used in a map” (Simon Poole). That’s why the folks over at Mapbox came up with the de-facto standard specification for encoding tiled vector data.

A common way to serve vector tiles from OpenStreetMap data is storing them in a single SQLite database, called MBTiles. A tile server will serve protobuf files with gridded vector data based on longitude, latitude and zoom level (translates to scale).

Take a look at the GIF below. Unique *.pbf files are stored (and requested by the client) in a schema that looks like this: path/to/data/zoom_level/x_position/y_position/xxx.pbf

Gridded structure of vector tiles
Gridded structure of vector tiles

The client, in particular a web map client (MapBox GL JS, MapLibre GL, OpenLayers) will render an image based on specific styling information, i.e. river geometries will be painted blue and 10 pixels wide, a bus station will get a special icon and so on. This separation of data and style is especially useful, because it means you can have one data source, and style it according to your needs.

There a lot of different solutions for producing vector tiles. PostGIS, the spatial extension for PostgreSQL, uses ST_AsMVT to return binary Mapbox Vector Tile (MVT) representations of geometries. This function in return is used by other tools:

More established open source map servers also support MVT output:

A couple of command line tools generate vector tiles from OpenStreetMap data, and here’s where it gets interesting. OpenMapTiles seems to be the most sophisticated solution, as it will also generate raster tiles through WMS and WMTS, but it requires a database. This is also the case for Baremaps. With a database in the background to store your OSM data, you can also quite easily update your data.

Command line tools requiring no database exist in the form of Planetiler and Tilemaker. Feed them an osm.pbf file and they will happily return MBTiles. Planetiler compares a little bit better in my opinion, because it makes better use of the available computing resources. That’s why I’m going to focus on Planetiler.

Planetiler is quite generous with it’s system requirements:

  • Java 16+ or Docker
  • at least 1GB of free disk space plus 5-10x the size of the .osm.pbf file
  • at least 0.5x as much free RAM as the input .osm.pbf file size

For a small extract, I don’t think anyone will be getting into trouble looking at these numbers.

I prefer using Docker. That way, I don’t have the hassle of installing a JDK and potentially destroying any PATH variables. Run the following command from your working directory.

docker run \
   -e JAVA_TOOL_OPTIONS="-Xmx1g" \
   -v "$(pwd)/data":/data \
   ghcr.io/onthegomap/planetiler:latest --download --area=brandenburg

Use the --download flag to let Planetiler handle the OSM file’s download. With the --area= flag, you can control which extract to download. If you’ve already downloaded an extract, use the --osm-path=path/to/file.osm.pbf flag. If you are using Docker, make sure to put that file into your volume used by Docker.

Once Planetiler is finished, you should have a file called output.mbtiles in your data directory. Let’s take a look at it!

Looking at this list you can see there are a lot of options on how to serve vector tiles, ranging from implementations in PHP and Node.js to Go, Python and others.

My go to solution is Tileserver GL, because it can also serve raster tiles as WMTS. By default it picks up any *.mbtiles file from the directory it is run from. Again, run it using Docker:

docker run \
   -v "$(pwd)/data":/data \
   -p "8080:8080" \
   maptiler/tileserver-gl --port 8080 --verbose

Navigate to http://localhost:8080/ to view the output of Planetiler.

Tileserver GL overview
Tileserver GL overview

Click the TileJSON link to see some metadata about the MBTiles. Save the value of the "bounds" key for later.

{
  "bounds": [
      11.22404, 
      51.35252, 
      14.77917, 
      53.5784
    ]
}

By default, Tileserver GL uses a “Basic preview” style. Let’s add one more style.

Planetiler creates vector tiles based on the OpenMapTiles profile. This is basically a schema on how to process OpenStreetMap data and which tags to incorporate into the MBTiles. If you want to create a style sheet for these vector tiles, you need to know how to reference streets, POIs, landmarks and so on. This is stored in the OpenMapTiles profile. Luckily, OpenMapTiles also provides predefined styles based on their profile. A style similar to OpenStreetMap is OSM Bright.

We need to download the style and make it available to Tileserver GL. Additionally, some fonts are required for this style.

# Download the style and extract it to data/styles/openmaptiles/osm-bright-gl-style
wget https://github.com/openmaptiles/osm-bright-gl-style/releases/download/v1.9/v1.9.zip
mkdir -p data/styles/openmaptiles/osm-bright-gl-style
unzip v1.9.zip -d data/styles/openmaptiles/osm-bright-gl-style

# Download fonts and extract them to data/fonts
wget https://github.com/openmaptiles/fonts/releases/download/v2.0/v2.0.zip
mkdir -p data/fonts
unzip v2.0.zip -d data/fonts

Next, create a file called config.json inside your data directory, with the following contents. Replace the value of "bounds" with the one you saved earlier.

{
  "options": {
    "paths": {
      "root": "/data",
      "fonts": "fonts",
      "styles": "styles",
      "mbtiles": "/data"
    }
  },
  "styles": {
    "osm-bright-gl-style": {
      "style": "openmaptiles/osm-bright-gl-style/style-local.json",
      "tilejson": {
        "bounds": [11.22404, 51.35252, 14.77917, 53.5784]
      }
    }
  },
  "data": {
    "v3": {
      "mbtiles": "output.mbtiles"
    }
  }
}

Run Tileserver GL again, with an additional config flag.

docker run \
   -v "$(pwd)/data":/data \
   -p "8080:8080" \
   maptiler/tileserver-gl --port 8080 --config /data/config.json --verbose --mbtiles output.mbtiles

Your new shiny OSM Bright style should be available.

OSM Bright GL Style
OSM Bright GL Style

OpenMapTiles provides more styles, i.e. the dark Fiord Color GL Style, which is best suited for data visualizations. Once downloaded, you can add them in your config.json and they will be available as additional styles.

It is quite possible to create background imagery for a Google Maps clone solely with open data and open source software tools. OpenStreetMap can provide the data, and modern rendering technologies (Web GL) allow for smooth and sharp web maps to be displayed in your browser. The presented tools make up only a fraction of the open source eco system that has been building up around creating and serving vector tiles. I recommend trying out other approaches.

OpenStreetMap Vector Tiles compared to Google Maps
OpenStreetMap Vector Tiles compared to Google Maps

Of course, creating background imagery is not enough, you need some interaction with the data. The next part will cover setting up a custom search tool for our map, also called geocoder.


Update 2022-06-21

In a previous version of this post I wrote OSM files contained “obsolete information” for web map rendering. As Simon Poole correctly pointed out, my intention was to say OSM files contain more information than is reasonable for usage in a web map.


Sources:


This series

  1. Creating a Google Maps Clone - Part 1 - Tiles and Styles
  2. Creating a Google Maps Clone - Part 2 - Geocoding or 'Where's that place?'
  3. Creating a Google Maps Client - Part 3 - Directions To Go

Share Social Interact With This Post At: Logo Mastodon Mastodon Logo Twitter Twitter
Calendar Posted:
Refresh Last Updated:
Person Posted By:
Folder Open Categories: Coding Tutorials Spatial Stuff

Responses

The first entry in a series dedicated to creating an entire clone of Google Maps, from creating vector mosaics to building a frontend GUI to connect everything, but for a limited spatial extension #stackhttps://chringel.dev/2022/06/creating-a-google-...

The first entry in a series dedicated to creating an entire clone of Google Maps, from creating vector mosaics to building a frontend GUI to connect everything, but for a limited spatial extension #stack chringel.dev/2022/06/creati…

La primera entrada d'una sèrie dedicada a crear un clon sencer de Google Maps, des de crear mosaics vectorials fins a construir una GUI de frontend per connectar-ho tot, però per a una extensió espacial limitada chringel.dev/2022/06/creati…

Yes, you're absolutely right. I'll update that section. Thanks for reading it!

Nitpicking ... "... because it contains a lot of obsolete information. " "obsolete" is likely not what you really wanted to say, but more "... because it contains far more information than we can reasonably use in a map"

https://chringel.dev/2022/06/creating-a-google-maps-clone-part-1-tiles-and-styles/#vectortiles #tileservergl #mapbox #maplibregl mapbox maplibregl tileservergl vectortiles Creating a Google Maps Clone - Part 1 - Tiles and Styles


Reactions