Skip to content

Sync Open Location Hub docs into website #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jillesvangurp merged 1 commit into from
Apr 2, 2026
Merged

Sync Open Location Hub docs into website #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 35 additions & 2 deletions content/open-location-hub/docs/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,42 @@
---
title: "Software Documentation"
description: "Reference documentation for hub architecture, configuration, authentication, and RPC behavior."
description: "Reference documentation for local setup, hub architecture, configuration, authentication, RPC behavior, and connector development."
draft: false
generated: true
generated_from: "docs/index.md"
github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/index.md"
---
Reference documentation for hub architecture, configuration, authentication, and RPC behavior.
_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._

Reference documentation for local setup, hub architecture, configuration,
authentication, RPC behavior, and connector development.

Start here if you want the laptop-friendly local runtime:

- [`docs/getting-started.md`](/open-location-hub/docs/getting-started/)

Core hub docs:

- [`docs/architecture.md`](/open-location-hub/docs/architecture/)
- [`docs/configuration.md`](/open-location-hub/docs/configuration/)
- [`docs/auth.md`](/open-location-hub/docs/auth/)
- [`docs/rpc.md`](/open-location-hub/docs/rpc/)

Connector docs:

- [`docs/connectors.md`](/open-location-hub/docs/connectors/)
- [`docs/connectors-websocket.md`](/open-location-hub/docs/connectors-websocket/)
- [`docs/connectors-mqtt.md`](/open-location-hub/docs/connectors-mqtt/)

Connector demonstrators live outside the hub runtime under
[`connectors/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors).
Shared connector-agnostic utility scripts live under
[`scripts/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/scripts).
The shared local runtime is documented in
[`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md).
Connector examples currently include
[`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md)
and
[`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md),
plus
[`connectors/replay/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/replay/README.md).
25 changes: 21 additions & 4 deletions content/open-location-hub/docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
---
title: "Architecture"
description: "System structure, processing flows, and trust boundaries."
description: "Generated documentation page for Architecture."
draft: false
generated: true
generated_from: "docs/architecture.md"
github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/architecture.md"
---
_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._

## Layers
- `cmd/hub`: process bootstrap and wiring
- `internal/config`: environment-driven configuration
Expand Down Expand Up @@ -33,17 +35,32 @@ github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/

## Event Fan-Out
1. REST, MQTT, or WebSocket ingest enters the shared hub service.
2. The hub validates, normalizes, deduplicates, updates in-memory transient state, and derives follow-on events.
3. The hub emits normalized internal events for locations, proximities, trackable motions, fence events, optional collision events, and metadata changes.
4. MQTT and WebSocket consume that same event stream and publish transport-specific payloads.
2. The hub validates, normalizes, deduplicates, and updates in-memory transient state on the ingest path.
3. A buffered native-publication stage emits native location and motion events without blocking ingest on downstream fan-out.
4. A second buffered decision stage is the insertion point for future filtered or smoothed track processing and currently drives alternate-CRS publication, geofence evaluation, and optional collision evaluation.
5. MQTT and WebSocket consume the resulting internal event stream and publish transport-specific payloads in batches.
6. When any non-critical queue fills, the hub drops newer work on that path rather than backpressuring raw ingest.

Implications:
- ingest logic is shared across REST, MQTT, and WebSocket
- MQTT is no longer the only downstream publication path
- the internal event seam decouples downstream publication from MQTT-specific topics
- location ingest latency is protected from slower transport fan-out, geofence work, or collision work
- the decision-stage queue is the intended insertion point for future filtered or smoothed track processing before fence/collision decisions
- WebSocket fan-out coalesces multiple internal events into fewer wrapper messages and drops outbound payloads for slow subscribers instead of tearing the connection down immediately
- hub-issued UUIDs for REST-managed resources, derived fence/collision events, and RPC caller IDs now use UUIDv7 so emitted identifiers are time-sortable
- internal hub events carry the persisted `origin_hub_id` so downstream transports preserve source provenance

## Observability Boundaries
- `internal/observability` owns OpenTelemetry resource setup, OTLP exporters, lifecycle management, and the small internal instrumentation API used by the rest of the runtime.
- OTLP export is collector-first: metrics, traces, and logs go to an OpenTelemetry collector such as SigNoz rather than to a hub-owned scrape endpoint.
- Transport handlers attach ingest transport context at entry so REST, MQTT, and WebSocket traffic create one shared root ingest span shape before entering `internal/hub`.
- The hot path records low-cardinality metrics for accepted, deduplicated, rejected, and failed ingest, queue depth and wait time, stage latency, event-bus fan-out, MQTT publish, WebSocket dispatch, RPC execution, and end-to-end processing time.
- Child spans isolate proximity resolution, native publication, decision processing, fence evaluation, collision evaluation, metadata reconcile, auth, and runtime dependency work so slow stages can be inspected directly.
- Asset-, provider-, zone-, and fence-centric identifiers belong on traces and structured logs only. They are intentionally excluded from normal metric labels so dashboards remain queryable under sustained ingest volume.
- Zap remains the application logging API. When OTLP logs are enabled, the logger tees into the OpenTelemetry bridge so the same structured events still appear locally while also being exported to the collector.
- Runtime gauges for queue occupancy, event-bus subscribers, and WebSocket connections are exposed from `internal/hub` through observable instruments so the e2e stack can dashboard degraded states without adding lock-heavy bookkeeping to the ingest path.

## RPC Control Plane
1. A client calls `GET /v2/rpc/available` or `PUT /v2/rpc` over HTTP.
2. REST auth verifies the bearer token and route-level access.
Expand Down
2 changes: 2 additions & 0 deletions content/open-location-hub/docs/auth.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ generated: true
generated_from: "docs/auth.md"
github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/auth.md"
---
_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._

This project supports standards-based JWT bearer authentication for the REST API and an authorization model built around JWT claims plus a server-side permissions file.

The same token verifier is also used for the OMLOX WebSocket surface, but WebSocket authentication happens per message through `params.token` instead of the HTTP `Authorization` header.
Expand Down
47 changes: 44 additions & 3 deletions content/open-location-hub/docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ generated: true
generated_from: "docs/configuration.md"
github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/configuration.md"
---
_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._

All runtime configuration is environment-driven.

Runtime lifecycle behavior:
Expand All @@ -23,7 +25,12 @@ Runtime lifecycle behavior:
- `POSTGRES_URL` (default `postgres://postgres:postgres@localhost:5432/openrtls?sslmode=disable`)
- `MQTT_BROKER_URL` (default `tcp://localhost:1883`)
- `WEBSOCKET_WRITE_TIMEOUT` (duration, default `5s`)
- `WEBSOCKET_OUTBOUND_BUFFER` (default `32`)
- `WEBSOCKET_READ_TIMEOUT` (duration, default `1m`)
- `WEBSOCKET_PING_INTERVAL` (duration, default `30s`)
- `WEBSOCKET_OUTBOUND_BUFFER` (default `256`)
- `EVENT_BUS_SUBSCRIBER_BUFFER` (default `1024`)
- `NATIVE_LOCATION_BUFFER` (default `2048`)
- `DERIVED_LOCATION_BUFFER` (default `1024`)

Hub metadata bootstrap behavior:
- the hub persists one durable metadata row in Postgres containing the stable `hub_id` and operator-facing label
Expand Down Expand Up @@ -55,9 +62,14 @@ Stateful ingest behavior:
- latest provider-source location state, trackable latest-location state, proximity hysteresis state, fence membership state, and collision pair state are all kept in process memory with the configured expiry semantics
- metadata is loaded from Postgres at startup, updated immediately after successful CRUD writes, and reconciled in the background every `METADATA_RECONCILE_INTERVAL`
- durable hub metadata is also loaded from Postgres at startup before the service begins accepting traffic
- WebSocket delivery uses a per-connection outbound queue capped by `WEBSOCKET_OUTBOUND_BUFFER`; slow subscribers are disconnected instead of backpressuring the ingest path
- WebSocket delivery uses a per-connection outbound queue capped by `WEBSOCKET_OUTBOUND_BUFFER`; when that queue fills, outbound payloads are dropped instead of backpressuring ingest or disconnecting the subscriber
- WebSocket liveness uses server ping frames every `WEBSOCKET_PING_INTERVAL` and considers the connection stale when no inbound message or pong arrives before `WEBSOCKET_READ_TIMEOUT`
- internal event-bus subscribers such as MQTT and WebSocket consume behind `EVENT_BUS_SUBSCRIBER_BUFFER`
- native location publication is queued behind `NATIVE_LOCATION_BUFFER` so ingest can decouple from transport fan-out
- post-native decision work such as future filtering, alternate-CRS publication, geofence evaluation, and collision evaluation is queued behind `DERIVED_LOCATION_BUFFER`
- when the native, decision, event-bus, or outbound socket queues are full, the hub drops newer work on those non-critical paths instead of slowing raw location ingest
- the `metadata_changes` WebSocket topic emits lightweight `{id,type,operation,timestamp}` notifications for zone, fence, trackable, and location-provider CRUD or reconcile drift
- when `COLLISIONS_ENABLED=true`, the hub evaluates trackable-versus-trackable collisions from the latest WGS84 motion state and keeps short-lived collision pair state in memory for `COLLISION_STATE_TTL`
- when `COLLISIONS_ENABLED=true`, the hub evaluates trackable-versus-trackable collisions from the latest active WGS84 motion state and keeps short-lived collision pair state in memory for `COLLISION_STATE_TTL`
- `COLLISION_COLLIDING_DEBOUNCE` limits repeated `colliding` emissions for already-active pairs

RPC behavior:
Expand Down Expand Up @@ -98,6 +110,35 @@ Proximity resolution behavior:

See [docs/auth.md](/open-location-hub/docs/auth/) for the full auth model, Dex setup, and permission file format.

## Telemetry
- `OTEL_ENABLED` (`true`/`false`, default `false`)
- `OTEL_METRICS_ENABLED` (`true`/`false`, default `true` when telemetry is enabled)
- `OTEL_TRACES_ENABLED` (`true`/`false`, default `true` when telemetry is enabled)
- `OTEL_LOGS_ENABLED` (`true`/`false`, default `true` when telemetry is enabled)
- `OTEL_EXPORTER_OTLP_ENDPOINT` (base OTLP HTTP endpoint such as `http://localhost:4318`)
- `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` (optional full traces endpoint override)
- `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` (optional full metrics endpoint override)
- `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` (optional full logs endpoint override)
- `OTEL_EXPORTER_OTLP_HEADERS` (comma-separated `key=value` headers for collector auth or routing)
- `OTEL_EXPORTER_OTLP_INSECURE` (`true`/`false`, default `false`)
- `OTEL_EXPORTER_OTLP_TIMEOUT` (duration, default `10s`)
- `OTEL_METRIC_EXPORT_INTERVAL` (duration, default `30s`)
- `OTEL_METRIC_EXPORT_TIMEOUT` (duration, default `10s`)
- `OTEL_TRACE_SAMPLE_RATIO` (number between `0` and `1`, default `1`)
- `OTEL_SERVICE_NAME` (default `open-rtls-hub`)
- `OTEL_SERVICE_VERSION` (optional override for release tagging)
- `OTEL_DEPLOYMENT_ENVIRONMENT` (optional deployment environment label such as `local-demo` or `production`)
- `OTEL_DEBUG_IDENTIFIERS` (`true`/`false`, default `false`)

Telemetry behavior:
- the hub exports OTLP metrics, traces, and logs directly to a collector and does not expose a separate Prometheus scrape endpoint in this slice
- `service.name`, `service.version`, `deployment.environment`, and the persisted `hub.id` are attached to the OTel resource when available
- invalid telemetry configuration fails startup only when telemetry is enabled
- metrics are intentionally low-cardinality and are labeled only with bounded dimensions such as transport, signal type, stage, feature, and outcome
- entity identifiers such as `trackable_id`, `provider_id`, `zone_id`, `fence_id`, and collision pair identifiers are emitted on spans and structured logs for drill-down, not on normal metric series
- runtime metrics cover ingest acceptance and deduplication, end-to-end processing latency, queue occupancy and wait time, fence evaluation, collision evaluation, MQTT/WebSocket publication, metadata reconcile, auth, and RPC outcomes
- the local demo stack under [`local-hub/`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub) enables all three OTLP signals by default and routes them to SigNoz

## RPC Security Defaults

For production deployments:
Expand Down
147 changes: 147 additions & 0 deletions content/open-location-hub/docs/connectors-mqtt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
---
title: "Connector Guide: MQTT"
description: "Use MQTT when your deployment already centers on a broker, when trusted edge adapters naturally publish broker messages, or when broker-based routing fits better than a long-lived WebSocket client connection."
draft: false
generated: true
generated_from: "docs/connectors-mqtt.md"
github_url: "https://github.com/Open-Location-Stack/open-location-hub/blob/main/docs/connectors-mqtt.md"
---
_This page is generated from the Open Location Hub source documentation and should not be edited in the website repository._

Use MQTT when your deployment already centers on a broker, when trusted edge
adapters naturally publish broker messages, or when broker-based routing fits
better than a long-lived WebSocket client connection.

This repository ships a concrete MQTT connector example in
[`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md).
The MQTT connector path is documented here against the hub's implemented OMLOX
MQTT surface. Transport details live in
[`specifications/omlox/mqtt.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/specifications/omlox/mqtt.md).

## When To Choose MQTT

Choose MQTT when:

- your source system already publishes to a broker
- you are running trusted local or edge adapters near devices
- you want broker-native fan-out, retention, or topic-based routing
- WebSocket is not the natural operational fit for the connector process

For user-facing applications and general connector examples, WebSocket is often
the simpler starting point. See
[docs/connectors-websocket.md](/open-location-hub/docs/connectors-websocket/).

## Required Inputs

An MQTT-based connector usually needs:

- `HUB_HTTP_URL`: REST base URL for optional metadata bootstrap
- `MQTT_BROKER_URL`: broker URL such as `tcp://localhost:1883`
- MQTT client identity and auth settings that fit your deployment
- `HUB_TOKEN` only if the connector also uses REST against an auth-enabled hub

The hub's runtime broker URL is documented in
[`docs/configuration.md`](/open-location-hub/docs/configuration/).

## Minimal Runtime Flow

The connector shape is almost the same as the WebSocket path:

1. Load environment variables.
2. Optionally use REST to ensure the provider and other metadata exist.
3. Connect to the MQTT broker.
4. Map upstream data to OMLOX payloads.
5. Publish each payload to the correct OMLOX ingest topic.
6. Optionally subscribe to hub-generated output topics for validation.
7. Reconnect and retry when broker connectivity fails.

The main difference is transport mapping. Instead of sending an OMLOX wrapper,
the connector publishes the payload body directly on the correct topic.

## Inbound Versus Outbound Topics

MQTT connector authors need to keep one boundary clear:

- inbound ingest topics are written by the connector and consumed by the hub
- outbound topics are written by the hub after it processes or derives data

Examples from the existing MQTT mapping:

- connector writes location ingest to
`/omlox/json/location_updates/pub/{provider_id}`
- connector writes proximity ingest to
`/omlox/json/proximity_updates/{source}/{provider_id}`
- hub writes processed locations to
`/omlox/json/location_updates/local/{provider_id}` and
`/omlox/json/location_updates/epsg4326/{provider_id}`
- hub writes derived events to topic families such as `fence_events`,
`trackable_motions`, and `collision_events`

Do not publish directly to the hub-generated output topics. Those exist so the
hub can publish normalized or derived results after it processes the inbound
payload.

## Topic Selection

The correct topic depends on the payload type:

- publish `Location` ingest to
`/omlox/json/location_updates/pub/{provider_id}`
- publish `Proximity` ingest to
`/omlox/json/proximity_updates/{source}/{provider_id}`
- consume processed and derived hub output from the output topic families when
you need validation or downstream processing

Use
[`specifications/omlox/mqtt.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/specifications/omlox/mqtt.md)
as the source of truth for topic families and payload expectations.

## Example Ingest Shape

For MQTT, the message body is the normalized OMLOX object itself. For example,
a connector publishing a location update would send a `Location` JSON payload to
`/omlox/json/location_updates/pub/{provider_id}`:

```json
{
"position": { "type": "Point", "coordinates": [8.5705, 50.0333] },
"crs": "EPSG:4326",
"provider_id": "my-connector",
"provider_type": "virtual",
"source": "upstream:object-123"
}
```

## Reusing The Bundled Connector Pattern

Even though the transport changes, the overall connector design can stay the
same as the bundled examples:

- keep env-driven configuration
- keep a small REST helper for metadata bootstrap
- keep the upstream polling or subscription loop separate from the transport
client
- keep mapping code focused on OMLOX payload construction

These repository examples are the best templates for that structure:

- [`connectors/gtfs/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/gtfs/README.md):
includes both a WebSocket connector and an MQTT connector that publish the
same normalized GTFS-derived `Location` payloads
- [`connectors/opensky/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/connectors/opensky/README.md)

The GTFS project is now the direct MQTT example. The OpenSky project remains a
useful example for shared project layout, env handling, and REST bootstrap.

## Local Run Path

For local development, use the shared demo stack:

```bash
local-hub/start_demo.sh
```

That stack includes the hub, Postgres, Dex, and Mosquitto. See
[`local-hub/README.md`](https://github.com/Open-Location-Stack/open-location-hub/blob/main/local-hub/README.md)
for the local runtime details and the token helper if your connector also uses
REST against an auth-enabled hub.
Loading
Loading