Add compose stack and CI

Author: ogrun

.github/workflows/ci.yml

@@ -0,0 +1,35 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install
+        run: pip install -e .
+      - name: Unit tests
+        run: python -m unittest discover -s tests
+
+  docker:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Build image
+        run: docker build -t ofmx2pgsql .
+      - name: Dry-run import in container
+        run: |
+          docker run --rm \
+            -e PG_DSN="postgresql://dummy:dummy@localhost:5432/dummy" \
+            -e OFMX_URL="https://snapshots.openflightmaps.org/live/2513/ofmx/lkaa/latest/ofmx_lk.zip" \
+            -e PG_SCHEMA="ofmx" \
+            -e APPLY_MIGRATIONS="false" \
+            -e DRY_RUN="true" \
+            -e VERBOSE="false" \
+            ofmx2pgsql

Makefile

@@ -0,0 +1,15 @@
+.PHONY: docker-build docker-import docker-compose-import
+
+docker-build:
+	docker build -t ofmx2pgsql .
+
+docker-import:
+	docker run --rm \
+		-e PG_DSN="${PG_DSN}" \
+		-e OFMX_URL="${OFMX_URL}" \
+		-e PG_SCHEMA="${PG_SCHEMA}" \
+		-e APPLY_MIGRATIONS="${APPLY_MIGRATIONS}" \
+		ofmx2pgsql
+
+docker-compose-import:
+	docker compose up --build --abort-on-container-exit

PROGRESS.md

@@ -17,6 +17,7 @@
 - INI-based config support added for CLI defaults.
 - Schema selection and JSON validation output added to the CLI.
 - Airspace uniqueness relaxed to preserve duplicate OFMX IDs.
+- Docker Compose stack added with a custom PostGIS image for arm64 hosts.
 - Implementation work has not started (no parser, schema, or imports yet).
 
 ## Milestones

README.md

@@ -9,6 +9,7 @@ Importer and schema are functional for the LK sample dataset. See `TODO.md` for
 - `pip install -e .` installs the package and the `psycopg` dependency.
 - `config/ofmx2pgsql.example.ini` shows the supported config keys.
 - `Dockerfile` builds a container that downloads a snapshot and imports it into PostGIS.
+- `docker-compose.yml` provides a PostGIS + importer stack for local testing.
 
 ## Repository Layout
 - `ofmx_lk/` sample OFMX data and reference materials.
@@ -50,6 +51,15 @@ docker run --rm \\
   ofmx2pgsql
 ```
 
+## Docker Compose
+Spin up a local PostGIS instance and run a one-shot import.
+
+```sh
+docker compose up --build --abort-on-container-exit
+```
+
+The compose stack builds a lightweight Postgres 18 + PostGIS image from `docker/postgis.Dockerfile` to support arm64 hosts.
+
 ## Data Notes
 The sample data in `ofmx_lk/` is treated as reference input. Avoid editing these files unless intentionally updating fixtures.
 

TODO.md

@@ -1,136 +1,31 @@
 # ofmx2pgsql – TODO List
 
-Build a lightweight importer for OpenFlightMaps OFMX data into PostgreSQL/PostGIS.
-
----
-
-## Phase -1 — Repository Setup
-
-- [x] Add contributor guidelines (`AGENTS.md`)
-- [x] Add project overview (`README.md`)
-- [x] Add progress log (`PROGRESS.md`)
-- [x] Add base ignore rules (`.gitignore`)
-
----
-
-## Phase 0 — Scope & Principles
-
-- [ ] Define non-goals
-  - [ ] No rendering optimizations
-  - [ ] No tile generation
-  - [ ] No attempt to support all OFMX edge cases initially
-- [ ] Decide target CRS
-  - [ ] Use EPSG:4326 only
-- [ ] Decide geometry fidelity
-  - [ ] Points and polygons only (no arcs initially)
-- [ ] Define vertical dimension handling
-  - [ ] Store altitudes as attributes (`lower_ft`, `upper_ft`)
-  - [ ] No 3D geometries in v1
-
----
-
-## Phase 1 — Understand OFMX Structure
-
-- [ ] Obtain OFMX XSD schema
-- [ ] Identify top-level entities
-  - [ ] Airports
-  - [ ] Runways
-  - [ ] RunwayEnds
-  - [ ] Airspaces
-  - [ ] Navaids
-  - [ ] Waypoints
-- [ ] Map ID relationships
-  - [ ] Airport ↔ Runway
-  - [ ] Runway ↔ RunwayEnd
-- [ ] Identify coordinate representations
-  - [ ] Lat/Lon formats
-  - [ ] Polygon definitions
-  - [ ] Circles / arcs
-- [ ] Document mandatory vs optional fields
-
----
-
-## Phase 2 — Define PostGIS Schema
-
-- [x] Design core tables
-  - [ ] airports
-  - [ ] runways
-  - [ ] runway_ends
-  - [ ] airspaces
-  - [ ] navaids
-  - [ ] waypoints
-- [x] Define primary keys
-- [x] Store OFMX source IDs
-- [x] Define foreign keys
-- [x] Define geometry columns
-  - [x] POINT for airports, navaids, waypoints
-  - [x] LINESTRING for runways, POINT for runway ends
-  - [x] MULTIPOLYGON for airspaces
-- [x] Add metadata fields
-  - [ ] source
-  - [ ] cycle
-  - [ ] valid_from
-  - [ ] valid_to
-- [x] Create spatial indexes (GIST)
-
----
-
-## Phase 3 — Project Skeleton
-
-- [x] Create repository `ofmx2pgsql`
-- [x] Choose language
-  - [x] Python
-- [ ] Setup virtual environment
-- [x] Setup CLI framework (`argparse` or `typer`)
-- [x] Define project structure
-
----
-
-## Phase 4 — XML Parsing Layer
-
-- [x] Add parser module scaffold
-- [x] Implement streaming XML parser (`xml.etree.ElementTree.iterparse`)
-- [x] Implement entity parsers
-- [x] Normalize coordinates to decimal degrees
-- [x] Preserve OFMX IDs
-
----
-
-## Phase 5 — Geometry Construction
-
-- [ ] Convert coordinates to Shapely geometries
-- [ ] Validate geometries
-- [ ] Parse altitude limits
-
----
-
-## Phase 6 — Database Writer
-
-- [x] Implement database connection
-- [x] Implement bulk inserts
-- [x] Implement idempotent loads
-
----
-
-## Phase 7 — CLI Features
-
-- [x] Import command
-- [x] Schema selection
-- [x] Dry run
-- [x] Verbose logging
-- [x] Config file defaults
-
----
-
-## Phase 8 — Validation & QA
-
-- [x] Compare counts with source
-- [x] JSON validation output
-- [ ] Visual inspection in QGIS
-
----
-
-## Phase 9 — Documentation
-
-- [ ] README
-- [ ] Example config
+Focused, remaining work for the OFMX-to-PostGIS importer.
+
+## Done (Milestones)
+- Repository bootstrapped (docs, config, tests, schema, CLI).
+- Parser and importer wired to LK sample data.
+- Import validation and dry-run workflows implemented.
+## Scope Decisions
+- Confirm non-goals (no rendering, no tile generation, limited edge-case support).
+- Lock CRS (EPSG:4326 only).
+- Confirm geometry fidelity (points, lines, polygons; arcs handled only via extension).
+- Decide altitude handling (attribute columns vs 3D geometries).
+
+## OFMX Understanding
+- Obtain and pin the OFMX XSD schema version used by the dataset.
+- Document mandatory vs optional fields for Ahp/Rwy/Rdn/Ase/Dpn/Ndb/Vor/Dme.
+- Clarify coordinate formats and arc/circle handling in core XML vs extensions.
+
+## Parsing & Geometry
+- Add geometry construction helpers (Shapely or raw WKT) for non-airspace shapes.
+- Validate geometries (self-intersections, ring closure, SRID consistency).
+- Parse altitude limits into numeric fields with normalized units.
+
+## Database & Import
+- Populate metadata fields (`source`, `cycle`, `valid_from`, `valid_to`) during import.
+- Decide how to version or archive imports across cycles.
+
+## Validation & QA
+- Visual inspection in QGIS for a sample import.
+- Add a lightweight regression dataset comparison (counts + spot checks).

docker-compose.yml

@@ -0,0 +1,26 @@
+services:
+  db:
+    build:
+      context: .
+      dockerfile: docker/postgis.Dockerfile
+    environment:
+      POSTGRES_USER: ofmx
+      POSTGRES_PASSWORD: ofmx
+      POSTGRES_DB: ofmx
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U ofmx -d ofmx"]
+      interval: 5s
+      timeout: 5s
+      retries: 5
+
+  importer:
+    build: .
+    depends_on:
+      db:
+        condition: service_healthy
+    environment:
+      PG_DSN: postgresql://ofmx:ofmx@db:5432/ofmx
+      OFMX_URL: https://snapshots.openflightmaps.org/live/2513/ofmx/lkaa/latest/ofmx_lk.zip
+      PG_SCHEMA: ofmx
+      APPLY_MIGRATIONS: "true"
+      VERBOSE: "true"

docker/postgis.Dockerfile

@@ -0,0 +1,8 @@
+FROM postgres:18-alpine
+
+RUN apk add --no-cache postgis \
+    && rm -rf /usr/local/share/postgresql/extension \
+    && ln -s /usr/share/postgresql18/extension /usr/local/share/postgresql/extension \
+    && ln -sf /usr/lib/postgresql18/postgis-3.so /usr/local/lib/postgresql/postgis-3.so \
+    && ln -sf /usr/lib/postgresql18/rtpostgis-3.so /usr/local/lib/postgresql/rtpostgis-3.so \
+    && ln -sf /usr/lib/postgresql18/postgis_sfcgal-3.so /usr/local/lib/postgresql/postgis_sfcgal-3.so