parseable-fly-metrics

Ship Fly.io platform metrics to Parseable. A small Go agent scrapes Fly's managed Prometheus federation endpoint for an entire org and forwards the samples to Parseable over prometheus_remote_write.

Run it as a dedicated Fly app, in your own infrastructure, or locally for development.

Requirements

• A Parseable deployment that exposes the POST /v1/prometheus/write ingestion route. This route is available in Parseable Enterprise / Prism (distributed) deployments.
• A Fly.io org with at least one running app. Apps that have stopped under auto_stop_machines = "stop" produce no metrics until they are woken.
• A Fly authentication token. flyctl auth token works; the agent sends it as Authorization: FlyV1 <token>, the scheme required by Fly's Prometheus endpoint.

Architecture

┌──────────────────────────── Fly.io org ─────────────────────────────┐
│                                                                     │
│   your-app-1     your-app-2     your-app-N                          │
│       │              │              │                               │
│       └──── metrics ─┴──────────────┘                               │
│                      │                                              │
│                      ▼                                              │
│       ┌─────────────────────────────────────────┐                   │
│       │  Fly managed Prometheus                 │                   │
│       │  api.fly.io/prometheus/<org>/federate   │                   │
│       └────────────────┬────────────────────────┘                   │
│                        │                                            │
│                        │ scrape every N seconds                     │
│                        │ Authorization: FlyV1 <token>               │
│                        │                                            │
│       ┌────────────────▼────────────────────────┐                   │
│       │ parseable-fly-metrics  (Fly app)        │   <-- THIS REPO   │
│       │   • parse Prom text exposition          │                   │
│       │   • encode prom_remote_write protobuf   │                   │
│       │   • snappy-compress + POST              │                   │
│       └────────────────┬────────────────────────┘                   │
└────────────────────────┼────────────────────────────────────────────┘
                         │
                         │ POST /v1/prometheus/write
                         │ basic auth, snappy protobuf
                         ▼
       ┌─────────────────────────────────────────┐
       │ Parseable Enterprise / Prism            │
       │   ingestor :8010  (write)               │
       │   query node :8000  (read / PromQL UI)  │
       └─────────────────────────────────────────┘

One agent runs per Fly org. The scrape selector is {app=~".+"}, so a single agent covers every app in the org — no changes to any app's own fly.toml.

Deploy to Fly

# 1) clone
git clone https://github.com/parseablehq/parseable-fly-metrics
cd parseable-fly-metrics

# 2) edit fly.toml — set `app = "<your-unique-name>"` and `primary_region`
#    (pick a region close to your Parseable ingestor)

# 3) create the app (no deploy yet)
flyctl apps create <your-unique-name> --org <your-fly-org>

# 4) set secrets
flyctl secrets set \
  FLY_ORG=<your-fly-org-slug> \
  FLY_METRICS_TOKEN=$(flyctl auth token) \
  PARSEABLE_URL=https://<your-parseable-ingestor>:8010 \
  PARSEABLE_USERNAME=<user> \
  PARSEABLE_PASSWORD=<pass>

# 5) deploy
flyctl deploy --ha=false

# 6) watch
flyctl logs

Expected log line per scrape interval:

scrape ok: families=61 series=134 samples=134 duration=460ms

The agent is push-only — no [[services]] block, no public port, no health check. Run a single machine; multiple instances would double-scrape.

Run locally

go build -o parseable-fly-metrics ./cmd/parseable-fly-metrics

FLY_ORG=<your-fly-org-slug> \
FLY_METRICS_TOKEN=$(flyctl auth token) \
PARSEABLE_URL=https://<your-parseable-ingestor>:8010 \
PARSEABLE_USERNAME=<user> \
PARSEABLE_PASSWORD=<pass> \
./parseable-fly-metrics

The binary is static; it runs anywhere with outbound HTTPS to api.fly.io and HTTP(S) to your Parseable ingestor.

Configuration

All configuration is via environment variables.

Variable	Required	Default	Notes
FLY_ORG	yes	—	Fly org slug. Find via flyctl orgs list.
FLY_METRICS_TOKEN	yes	—	Output of flyctl auth token or flyctl tokens create readonly -o <org>.
PARSEABLE_URL	yes	—	Base URL of the Parseable ingestor. In Prism deployments, point at an ingestor node, not the query node.
PARSEABLE_USERNAME	yes	—	HTTP basic auth username.
PARSEABLE_PASSWORD	yes	—	HTTP basic auth password.
PARSEABLE_STREAM	no	fly_metrics	Target stream name. Parseable creates the stream automatically on first write.
SCRAPE_INTERVAL_SECS	no	30	Seconds between scrapes.

Verify ingestion

Query the Parseable query node (not the ingestor):

curl -u <user>:<pass> -X POST \
  -H 'Content-Type: application/json' \
  -d '{"query":"select metric_name, app, region, data_point_value from fly_metrics order by p_timestamp desc limit 5","startTime":"1h","endTime":"now"}' \
  https://<your-parseable-query-node>:8000/api/v1/query

A populated stream looks like:

{
  "app": "hellofly",
  "region": "sin",
  "instance": "9080d161a52928",
  "host": "f5d5",
  "metric_name": "fly_instance_memory_buffers",
  "metric_type": "gauge",
  "data_point_value": 778240,
  "time_unix_nano": "2026-05-18T06:45:38.637",
  "scope_name": "prometheus_remote_write",
  "p_format": "otel-metrics"
}

Stream schema is inferred asynchronously; a query immediately after the first scrape may return []. Wait one scrape interval and retry.

How it works

1. Every SCRAPE_INTERVAL_SECS, the agent issues GET https://api.fly.io/prometheus/${FLY_ORG}/federate?match[]={app=~".+"} with Authorization: FlyV1 ${FLY_METRICS_TOKEN}.
2. The response (Prometheus text exposition format) is parsed into metric families. Fly's /federate returns samples without # TYPE directives; the agent treats them as UNTYPED.
3. Metric families are converted to a Prometheus remote_write WriteRequest protobuf, snappy-compressed, and POSTed to ${PARSEABLE_URL}/v1/prometheus/write with headers:
   • Content-Type: application/x-protobuf
   • Content-Encoding: snappy
   • X-P-Stream: ${PARSEABLE_STREAM}
   • X-P-Log-Source: otel-metrics
4. Parseable maps the remote_write payload to its internal OTLP-shaped metric rows; counters, gauges, histograms, and summaries all become queryable rows in the target stream.

Troubleshooting

Symptom	Cause / Fix
scrape error: ... HTTP 401: something went wrong resolving organization	FLY_METRICS_TOKEN or FLY_ORG is wrong. The agent sends Authorization: FlyV1 <token>; Bearer will fail. Verify the slug with flyctl orgs list.
scrape ok: families=0 series=0 samples=0	Every app in the org is idle (auto_stop_machines = "stop"). Wake an app by hitting its public URL — Fly emits metrics only for running machines.
write error: ... HTTP 404	PARSEABLE_URL points at a deployment without the /v1/prometheus/write route, or at a Prism query node. Use a Parseable Enterprise/Prism ingestor.
write error: ... HTTP 400: Snappy decompression failed	A request reached the route with malformed snappy. Check agent logs for upstream marshal errors.
Stream exists but select * limit 1 returns []	Parseable infers schema asynchronously after the first write. Retry after one scrape interval, or query with a wider startTime.

Notes

• Auth scheme. Fly's Prometheus endpoint accepts Authorization: FlyV1 <token> for tokens issued by recent flyctl versions (macaroons, fm2_…). The Bearer scheme returns HTTP 401 for these tokens.
• Idle apps. Apps with auto_stop_machines = "stop" stop emitting metrics when idle; /federate returns 200 with an empty body.