Skip to content

Add add_h3t_source() for tiled H3J (h3t) data#199

Open
bbest wants to merge 12 commits into
walkerke:mainfrom
bbest:feat/add-h3t-source
Open

Add add_h3t_source() for tiled H3J (h3t) data#199
bbest wants to merge 12 commits into
walkerke:mainfrom
bbest:feat/add-h3t-source

Conversation

@bbest
Copy link
Copy Markdown
Contributor

@bbest bbest commented Apr 21, 2026

Summary

Adds add_h3t_source() — an R wrapper for the h3tiles:// MapLibre protocol that's already bundled in inst/htmlwidgets/lib/h3j-h3t/h3j_h3t.js but previously had no R entry point. This lets users drive H3 hex layers from a tiled HTTP endpoint (one request per viewport) instead of loading a single monolithic H3J file up front.

While wiring this up I also hit five bugs in the bundled h3j-h3t JS that blocked it from working on MapLibre v3+/v4/v5 and from supporting multiple sources per page. Those are now fixed in the bundled copy (v0.9.7) and upstreamed in INSPIDE/h3j-h3t#8 — happy to swap the bundled file for INSPIDE's release once that PR lands.

Motivation

add_h3j_source() is great when your whole dataset fits in a single JSON file, but it breaks down once you have hundreds of thousands of cells or want to filter on the fly. The h3t tile protocol solves both — the browser only fetches cells inside the current viewport/zoom, and the HTTP layer becomes a natural cache key.

Representative use case (a Shiny app doing species × environment compare): preloading all 10 H3 resolutions of a 500 k-row DuckDB aggregate was taking ~10 s per session and couldn't cache user filter changes. Moving to add_h3t_source() dropped initial load to <2 s and made every filter change a server-side operation only.

What's included

R / widget wiring

  • add_h3t_source(map, id, tiles, sourcelayer, geometry_type, minzoom, maxzoom, promote_id, debug) in R/h3j-h3t.R. Supports both static maps and maplibre_proxy updates.
  • x.h3t_sources init handler in inst/htmlwidgets/maplibregl.js and inst/htmlwidgets/maplibregl_compare.js.
  • New add_h3t_sources proxy-message handler. Drive-by: also adds the missing add_h3j_sources proxy handler — add_h3j_source()'s proxy branch previously had no JS counterpart.
  • man/add_h3t_source.Rd with a runnable example.
  • Small defensive fixes in maplibregl_compare.js's layers_control: pre-check with map.getLayer(id) before calling getLayoutProperty / setLayoutProperty, since the latter emits an error event (not a throw) when the layer isn't on that map. In a compare widget a single layers_control can reference ids from both sides, so this check is necessary to avoid console noise like Cannot get style of non-existing layer "x".

Bundled h3j-h3t JS: 0.9.2 → 0.9.7

Upstreamed in INSPIDE/h3j-h3t#8. Summary:

version fix symptom
0.9.3 empty tiles return ArrayBuffer(0) instead of null Cannot read properties of undefined (reading 'data') on ocean tiles of coastal data
0.9.4 support MapLibre v3+/v4 Promise protocol API (in addition to v2 callback) e is not a function on every tile request
0.9.5 return ArrayBuffer instead of Uint8Array MapLibre v5 rejects the Uint8Array vt-pbf returns
0.9.6 parse z/x/y with a regex anchored on .h3t$ rather than splitting on / and . Any URL with a query string containing a dot (e.g. ?release=v2026.04.08) parsed NaN as x, every tile empty
0.9.7 mint a unique protocol scheme per source (h3t1://, h3t2:// …) instead of the fixed h3tiles:// map.addProtocol('h3tiles', handler) is a global registry — the second call overwrote the first, so two sources on one map (or one source on each side of a compare widget) clobbered each other's sourcelayer

Net: add_h3t_source() now works cleanly against MapLibre 3.6.x – 5.22.x, with multiple sources on a single page and on both sides of compare().

R CMD check passes with 0 errors / 0 warnings / 0 notes (with _R_CHECK_FORCE_SUGGESTS_=false to skip the optional mapboxapi dep).

Reproducible demo

A companion tile server lives at CalCOFI/api-h3t (MIT, standalone, pairs with any DuckDB file that has H3 cells). To reproduce:

# start an h3t tile server against your DuckDB
git clone https://github.com/CalCOFI/api-h3t.git && cd api-h3t
docker build -t api-h3t .
docker run --rm -p 8889:8889 \
  -e DUCKDB_PATH=/data/your.duckdb \
  -v "/path/to/dir:/data:ro" \
  api-h3t
# drive it from mapgl
library(mapgl)
library(base64enc)

sql <- "SELECT hex_h3res{{res}} AS cell_id, COUNT(*) AS value, COUNT(*) AS n
          FROM your_table GROUP BY 1"
q   <- base64encode(charToRaw(sql)) |>
         chartr("+/", "-_", x = _) |> sub("=+$", "", x = _)

maplibre(center = c(-119, 34), zoom = 5) |>
  add_h3t_source(
    id    = "cells",
    tiles = sprintf("h3tiles://localhost:8889/h3t/{z}/{x}/{y}.h3t?q=%s", q)
  ) |>
  add_fill_layer(
    id           = "cells",
    source       = "cells",
    source_layer = "cells",
    fill_color   = interpolate(column = "value",
                               values = c(1, 100),
                               stops  = c("#ffffcc", "#e31a1c")),
    fill_opacity = 0.7
  )

The {{res}} placeholder is a convention of the example server — add_h3t_source() itself is source-agnostic and will work with any endpoint that returns h3j JSON.

Scope / non-goals

  • Mapbox parity: not touched. mapboxgl.js has no addH3TSource symbol today (and no addH3JSource either). Happy to mirror if you'd like it in this PR.
  • Proxy setter for updating tile URLs after init: not included; re-calling add_h3t_source() on a maplibre_proxy adds a new source, it doesn't mutate an existing one. Worth a separate PR once a clear API emerges.

Test plan

  • R CMD check --no-manual → 0E / 0W / 0N
  • Interactive smoke: maplibre() |> add_h3t_source(...) |> add_fill_layer(...) renders; tiles requested on pan/zoom; MVT conversion happens client-side
  • Shiny proxy: maplibre_proxy("map") |> add_h3t_source(...) triggers add_h3t_sources message; source registers
  • Compare widget: compare(m1, m2) with h3t sources on both sides — both load and do not clobber each other (0.9.7 fix)
  • Exercised against two real datasets (~3 k cells @ res 5, ~500 k underlying rows) on MapLibre 5.22.x

🤖 Generated with Claude Code

Exposes the bundled h3j-h3t.js addH3TSource() method to R. Lets users
drive H3 hex layers from a tiled HTTP endpoint (one request per viewport)
instead of loading a single monolithic H3J file up front.

- R: add_h3t_source(map, id, tiles, sourcelayer, geometry_type, minzoom,
  maxzoom, promote_id, debug). Supports static maps and maplibre_proxy
  updates.
- JS init: x.h3t_sources handler in maplibregl.js and maplibregl_compare.js.
- JS proxy: add_h3t_sources message handler; drive-by fix for the missing
  add_h3j_sources handler (add_h3j_source()'s proxy path previously had no
  JS counterpart).

R CMD check passes 0E/0W/0N.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cboettig
Copy link
Copy Markdown
Contributor

Very nice!

bbest and others added 4 commits April 21, 2026 23:40
Upstream fix in bbest/h3j-h3t@v0.9.3: empty tiles (no H3 cells in the
requested bbox) now return an empty-but-valid MVT instead of throwing
through MapLibre as "Cannot read properties of undefined (reading 'data')".

This is mechanical — rebuilds the bundled JS and bumps the yaml version.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The compare widget has its own yaml — missed it in the previous bump.
This is what the CalCOFI int-app uses (dual-panel sp/env compare map),
which is why the browser was still loading h3j-h3t-0.9.2 after the
mapgl reinstall and missing the empty-tile fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bundled maplibre-gl in this package uses the promise-based
addProtocol signature (v3+). h3j-h3t <= 0.9.3 assumed the old
callback signature, so MapLibre was calling our handler with
(params, abortController) and we tried to invoke the
AbortController as a callback, throwing
"Uncaught (in promise) TypeError: e is not a function" which
MapLibre reported as "Cannot read properties of undefined
(reading 'data')".

bbest/h3j-h3t@0.9.4 auto-detects the signature and returns a
Promise<{data}> on v3+, keeping callback compat for v2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… map

In the compare widget each side only has its own layers, but the
layers_control may legitimately reference ids from the other side so
the toggle can fire on both maps simultaneously. The initial-visibility
check was calling getLayoutProperty() on every id without try/catch,
which threw "Cannot get style of non-existing layer" and halted
applyMapModifications mid-init — nothing that ran after that step (no
terrain, no controls, and more importantly the deeper parts of the flow)
could execute, leaving the map blank.

- Wrap the initial getLayoutProperty() in try/catch, defaulting to
  "visible".
- Same pattern for the click handler: read visibility from whichever
  map actually has the layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bbest
Copy link
Copy Markdown
Contributor Author

bbest commented Apr 21, 2026

Standby, this is not yet functional.

PS @cboettig this was inspired by your addition of add_h3j_source() #68, which I would not even attempt except for the superhuman powers of Claude to facilitate.

bbest and others added 7 commits April 22, 2026 00:40
…tion

Logs '[mapgl] maplibregl_compare.js build: <date>' to the console so
it's visible at a glance which version is actually loaded in the
browser vs what's on disk.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…yer)

So we can see which map (before/after) gets which sources and layers in
the compare widget, to diagnose why only one side renders hexagons.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
getLayoutProperty() on a missing layer fires an error event via
this.fire() rather than throwing, so the try/catch could not silence
the "Cannot get style of non-existing layer" console noise. Pre-check
with map.getLayer(id) in both the init visibility read and the
onclick toggle path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These were added to diagnose the per-side addProtocol collision in
h3j-h3t; that is fixed (v0.9.7 mints a unique scheme per source), so
the logs are noise now. Build marker retained for install verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bbest
Copy link
Copy Markdown
Contributor Author

bbest commented Apr 22, 2026

All is working now, but hope to update with an easier reproducible example in the next couple of weeks.

This update also depends on INSPIDE/h3j-h3t#8, and that JS library has not been updated in 4 years.

Here's the (somewhat complicated) code of where it's working using mapgl::compare() comparing biological sampling on left with environmental parameters on the right:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants