Add Overture data source support

[migurski]

Add basemap support for Overture Maps input data as alternative to OSM extracts. Port kind=/kind_detail=/min_zoom= mappings with no changes to MapLibre styles.

Layers

All Protomaps layers except for Boundaries and Transit have some coverage in this PR.

POIs

• Overture theme=places basic_category mapped to Protomaps kind, with limited exceptions
• No OSM-style polygon area grading is available for POI importance
• Future work:
  • Can we use QRank to find high-priority POIs?
  • Try filtering by Overture confidence scores to eliminate bad POIs

Roads

• Rendering includes theme=transportation type=segment subtype=road, subtype=rail, and subtype=water
• Overture class and subclass mapped to Protomaps kind, kind_detail, and internal highway
• Uses linear referencing from Overture array properties like road_flags to split linestrings on bridge, tunnel, and level flags to match rendering of OSM basemap
• No features for airport runways are available

Places

• Rendering includes high-zoom places like cities and neighborhoods from theme=places
• Future work:
  • Include low-zoom places like states and countries

Buildings

• Rendering includes both theme=buildings type=building and type=building_part to match Protomaps visual style

Landuse

• Rendering includes theme=base type=land_use to match Protomaps visual style

Earth, Water, and Land Cover

• Rendering uses theme=base type=land, type=water, type=land_cover to match Protomaps visual style
• Future work:
  • Reduce excessive labels displayed for water bodies
  • Fix visual appearance of grainy land cover at low zooms

Testing

Extract Overture data with e.g. DuckDB:

COPY (
    SELECT *
    FROM read_parquet(
        's3://overturemaps-us-west-2/release/2025-12-17.0/**/*.parquet',
        hive_partitioning=1, filename=1, union_by_name=1
    )
    WHERE theme IN ('transportation', 'places', 'base', 'buildings', 'divisions')
      AND bbox.xmin <= -121
      AND bbox.xmax >= -123
      AND bbox.ymin <= 38
      AND bbox.ymax >= 37
) TO 'data/sources/bay-area.parquet' (FORMAT PARQUET);

Generate PMTiles from Overture data:

java -jar target/protomaps-basemap-HEAD-with-deps.jar \
    --overture=data/sources/bay-area.parquet --download --force

Run the app/ map frontend or one of the HTML examples to preview.

Screenshots

Taken from interactive preview at mike.teczno.com; compare with OSM data at maps.protomaps.com.

Flavor Compatibility

The changes in this PR map Overture properties to existing Protomaps conventions with no changes to styles, so all five of the included flavors are compatible: black, grayscale, white, light, and dark.

[danabauer]

This is awesome, @migurski. /cc @jonahadkins

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
82.3% Coverage on New Code
1.7% Duplication on New Code

See analysis details on SonarQube Cloud

[bdon]

COPY (
    SELECT *
    FROM read_parquet(
        's3://overturemaps-us-west-2/release/2025-12-17.0/**/*.parquet',
        hive_partitioning=1, filename=1, union_by_name=1
    )
    WHERE theme IN ('transportation', 'places', 'base', 'buildings', 'divisions')
      AND bbox.xmin <= -121
      AND bbox.xmax >= -123
      AND bbox.ymin <= 38
      AND bbox.ymax >= 37
) TO 'data/sources/bay-area.parquet' (FORMAT PARQUET);

Curious, how long does this take for you to download? From how long it takes it seems to require some enumeration over the overture .parquet files

[wipfli]

I am just downloading the entire December release now with something like:

aws s3 cp --no-sign-request --recursive s3://overturemaps-us-west-2/release/2025-12-17.0/ .

With about 10 MiB/s it is a bit pedestrian but if the whole thing is only roughly 500 GB then the download should still complete within 14 hours. Sometimes those downloads also get faster over time. Let's see...

[wipfli]

Thanks for making the online demo @migurski! The first thing that I saw when opening it was the Emirate Kush Cargo Airport in the middle of Oakland. According to the overture explorer this POI has a confidence score of roughly 63 percent. To experiment with thresholding it might make sense to temporarily add the confidence score in the tiles and then we could add a slider to the frontend and change the maplibre style with a filter or so based on the slider value.

[migurski]

I am just downloading the entire December release now with something like:

aws s3 cp --no-sign-request --recursive s3://overturemaps-us-west-2/release/2025-12-17.0/ .

With about 10 MiB/s it is a bit pedestrian but if the whole thing is only roughly 500 GB then the download should still complete within 14 hours. Sometimes those downloads also get faster over time. Let's see...

Try the DuckDB sample code in the description! The OMS Parquet files are already clustered spatially, and small areas can take just minutes to extract.

[migurski]

COPY (
    SELECT *
    FROM read_parquet(
        's3://overturemaps-us-west-2/release/2025-12-17.0/**/*.parquet',
        hive_partitioning=1, filename=1, union_by_name=1
    )
    WHERE theme IN ('transportation', 'places', 'base', 'buildings', 'divisions')
      AND bbox.xmin <= -121
      AND bbox.xmax >= -123
      AND bbox.ymin <= 38
      AND bbox.ymax >= 37
) TO 'data/sources/bay-area.parquet' (FORMAT PARQUET);

Curious, how long does this take for you to download? From how long it takes it seems to require some enumeration over the overture .parquet files

For me, about a minute or two.

[migurski]

Thanks for making the online demo @migurski! The first thing that I saw when opening it was the Emirate Kush Cargo Airport in the middle of Oakland. According to the overture explorer this POI has a confidence score of roughly 63 percent. To experiment with thresholding it might make sense to temporarily add the confidence score in the tiles and then we could add a slider to the frontend and change the maplibre style with a filter or so based on the slider value.

I agree, and incorporating QRank and confidence would be a good way to improve the POI quality since we don't have way areas or building heights like in OSM.

[wipfli]

I was able to run a local build of Zürich:

[wipfli]

I have a local copy now of the roughly 500 GB overture data.
Is it possible to run this on the full Overture data planet-wide?

[wipfli]

@msbarry uses the following I think to read a parquet directory:

https://github.com/onthegomap/planetiler-examples/blob/8901f77284712fd8f8f35f3fface712962d6287c/Overture.java#L42-L46

[migurski]

I have a local copy now of the roughly 500 GB overture data. Is it possible to run this on the full Overture data planet-wide?

I have not tried yet at this scale, but I’m not sure it would be meaningfully different from a planet-scale OSM import.

[wipfli]

The ideal way to interact with Overture data from my perspective would be that you specify an area as defined by the Geofabrik polygons with --area=monaco for example and you say that you want Overture instead of OpenStreetMap by adding a flag like --use-overture or so.
If those arguments are present, our program should download the polygon for Monaco from Geofabrik and then get an extract from the Overture parquets. @msbarry can the parquet reader already ingest polygons for clipping? Or is it maybe possible with bboxes?

[wipfli]

If no --area flag is provided then it should just make the whole planet.

[msbarry]

The overture feature filter only works with a bounding box, although if you specify --polygon=country.poly it will initialize the bounding box to what covers that polygon and only process tiles that overlap the polygon.

Another possibility is to implement this feature on planetiler: onthegomap/planetiler#1455 which would let it just read the relevant parts of the parquet file over http while processing it so you don't need to store anything on disk. Overture just started publishing https://stac.overturemaps.org/ that would make this much simpler and faster.

[migurski]

I’m reluctant to formalize the Geofabrik area naming conventions for non-OSM data, but maybe there’s a way to do that cleanly by referencing the content of tiles/src/main/resources/borders.json ?

I do like the idea of referencing live Overture data and avoiding local storage. Perhaps in a followup PR?

[bdon]

I would prefer for now that the Overture developer experience is analogous to the OSM one - a single input file that you can acquire however you like. The DuckDB script that creates a single .parquet from remote Overture data seems ideal.

Making that work with named areas would need a gazetteer, I think that would be an interesting external project, maybe using Overture boundaries themselves with a cloud-native storage format>

[bdon]

It would be useful if planetiler.addParquetSource read the bounds of the Parquet file (is this standardized in GeoParquet?) and then passed it through to the bounds header of the tile archive output. This way "fit bounds" on maps.protomaps.com would zoom to the precise area.

[migurski]

It would be useful if planetiler.addParquetSource read the bounds of the Parquet file (is this standardized in GeoParquet?) and then passed it through to the bounds header of the tile archive output. This way "fit bounds" on maps.protomaps.com would zoom to the precise area.

I’ll research what’s needed for this in the current PR.

[migurski]

It would be useful if planetiler.addParquetSource read the bounds of the Parquet file (is this standardized in GeoParquet?) and then passed it through to the bounds header of the tile archive output. This way "fit bounds" on maps.protomaps.com would zoom to the precise area.

I’ll research what’s needed for this in the current PR.

The adjustment for this was small, implemented in 160a21c. The input Parquet files don’t consistently have bounding boxes defined when they're generated by DuckDB, so in my tests the output bounds are accurately set to match the whole-earth bbox of the input data. A way around this might be to read the actual envelope from Parquet row groups?

I resolved the merge conflicts.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
80.8% Coverage on New Code
1.6% Duplication on New Code

See analysis details on SonarQube Cloud