Sync Layer Server

---

Service that continuously synchronizes geospatial layer data from a third-party GraphQL API into a remote PostgreSQL database.

See SYNC.md for a detailed description of the sync module architecture and lifecycle.

Development

When in development you should use the command npm run start:dev. The main benefits are that it enables offline mode for the config package, and source map support for NodeJS errors.

Features:

• eslint configuration by @map-colonies/eslint-config

• prettier configuration by @map-colonies/prettier-config

• vitest

• .nvmrc

• Multi stage production-ready Dockerfile

• commitlint

• git hooks

• logging by @map-colonies/js-logger

• config load with @map-colonies/config

• Tracing and metrics by @map-colonies/telemetry

• github templates

• bug report

• feature request

• pull request

• github actions

• on pull_request

• LGTM

• test

• lint

• snyk

Installation

Install deps with npm

npm install

Run Locally

Clone the project

git clone https://link-to-project

Go to the project directory

cd sync-layer-server

Install dependencies

npm install

Start the server

npm run start

Run Migrations

Run migrations before you start the app

Migrations Development

• Update metadata file or change DB details (fakeDB for example)
• npm run migration:create

Shell

Run the following command:

npm run migration:run

Docker

Build the migrations image:

docker build -t sync-layer-server-migration:latest -f migrations.Dockerfile .

Run image:

docker run -it --rm --network host sync-layer-server-migration:latest

If you want to change the connection properties you can do it via either:

1. Env variables
2. Inject a config file based on your environment

Via env variables:

docker run -it -e DB_USERNAME=VALUE  -e DB_PASSWORD=VALUE -e DB_NAME=VALUE -e DB_TYPE=VALUE -e DB_HOST=VALUE -e DB_PORT=VALUE --rm --network host sync-layer-server-migration:latest

SSL/TLS Configuration

For secure database connections using SSL certificates:

Environment Variables:

• DB_ENABLE_SSL - Set to true to enable SSL (default: false)
• DB_SSL_KEY_PATH - Path to client private key file
• DB_SSL_CERT_PATH - Path to client certificate file
• DB_SSL_CA_PATH - Path to CA certificate file

Example with SSL:

docker run -it \
  -e DB_HOST=your-db-host \
  -e DB_PORT=5432 \
  -e DB_USERNAME=postgres \
  -e DB_PASSWORD=postgres \
  -e DB_NAME=sync-layer \
  -e DB_ENABLE_SSL=true \
  -e DB_SSL_KEY_PATH=/app/certs/client-key.pem \
  -e DB_SSL_CERT_PATH=/app/certs/client-cert.pem \
  -e DB_SSL_CA_PATH=/app/certs/ca.pem \
  -v /path/to/certs:/app/certs:ro \
  --rm --network host \
  sync-layer-server-migration:latest

Via injecting a config file, assuming you want to run the migration on your production:

production.json:

{
  "openapiConfig": {
    "filePath": "./openapi3.yaml",
    "basePath": "/docs",
    "rawPath": "/api",
    "uiPath": "/api"
  },
  "logger": {
    "level": "info"
  },
  "server": {
    "port": "8085"
  },
  "db": {
    "type": "postgres",
    "username": "postgres",
    "password": "postgres",
    "database": "catalog",
    "port": 5432
  }
}

docker run -it --rm -e NODE_ENV=production --network host -v /path/to/proudction.json:/usr/app/config/production.json sync-layer-server-migrations:latest

---

Running Tests

To run tests, run the following command

npm run test

To only run unit tests:

npm run test:unit

To only run integration tests:

npm run test:integration