Skip to content

OpenTelemetry: HTTP server request metrics #757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dahlia wants to merge 5 commits into
base: main
Choose a base branch
from
Open

OpenTelemetry: HTTP server request metrics #757

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b46c627
Add OpenTelemetry HTTP server metrics for Federation.fetch()
dahlia May 7, 2026
ce625ec
Recognize QUERY as a known HTTP method
dahlia May 7, 2026
f230826
Add explicit bucket boundaries to HTTP server duration
dahlia May 7, 2026
ffc7fd3
Type fedify.endpoint as a literal union
dahlia May 7, 2026
23a6491
Cover the user-defined collection collapse in tests
dahlia May 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,25 @@ To be released.
attributes, and `TraceActivityRecord.activityJson` is present only when the
span event includes full activity JSON. [[#316], [#619], [#755]]

- Added OpenTelemetry HTTP server metrics for inbound requests handled by
`Federation.fetch()`: `fedify.http.server.request.count` (Counter) and
`fedify.http.server.request.duration` (Histogram). Both instruments carry
`http.request.method`, `fedify.endpoint`, optional
`http.response.status_code`, and optional `fedify.route.template`
attributes so that operators can monitor aggregate request rate, latency,
and status-code error rate even when traces are sampled. Attributes
deliberately exclude raw URLs, query strings, and identifier values to
keep cardinality bounded. [[#316], [#736], [#757]]

[#316]: https://github.com/fedify-dev/fedify/issues/316
[#619]: https://github.com/fedify-dev/fedify/issues/619
[#735]: https://github.com/fedify-dev/fedify/issues/735
[#736]: https://github.com/fedify-dev/fedify/issues/736
[#748]: https://github.com/fedify-dev/fedify/pull/748
[#752]: https://github.com/fedify-dev/fedify/issues/752
[#753]: https://github.com/fedify-dev/fedify/pull/753
[#755]: https://github.com/fedify-dev/fedify/pull/755
[#757]: https://github.com/fedify-dev/fedify/pull/757

### @fedify/fixture

Expand Down
49 changes: 42 additions & 7 deletions docs/manual/opentelemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,13 +296,15 @@ Instrumented metrics

Fedify records the following OpenTelemetry metrics:

| Metric name | Instrument | Unit | Description |
| -------------------------------------------- | ---------- | ----------- | ----------------------------------------------------------- |
| `activitypub.delivery.sent` | Counter | `{attempt}` | Counts outgoing ActivityPub delivery attempts. |
| `activitypub.delivery.permanent_failure` | Counter | `{failure}` | Counts outgoing deliveries abandoned as permanent failures. |
| `activitypub.delivery.duration` | Histogram | `ms` | Measures outgoing ActivityPub delivery attempt duration. |
| `activitypub.inbox.processing_duration` | Histogram | `ms` | Measures inbox listener processing duration. |
| `activitypub.signature.verification_failure` | Counter | `{failure}` | Counts failed signature verification for inbox requests. |
| Metric name | Instrument | Unit | Description |
| -------------------------------------------- | ---------- | ----------- | --------------------------------------------------------------- |
| `activitypub.delivery.sent` | Counter | `{attempt}` | Counts outgoing ActivityPub delivery attempts. |
| `activitypub.delivery.permanent_failure` | Counter | `{failure}` | Counts outgoing deliveries abandoned as permanent failures. |
| `activitypub.delivery.duration` | Histogram | `ms` | Measures outgoing ActivityPub delivery attempt duration. |
| `activitypub.inbox.processing_duration` | Histogram | `ms` | Measures inbox listener processing duration. |
| `activitypub.signature.verification_failure` | Counter | `{failure}` | Counts failed signature verification for inbox requests. |
| `fedify.http.server.request.count` | Counter | `{request}` | Counts inbound HTTP requests handled by `Federation.fetch()`. |
| `fedify.http.server.request.duration` | Histogram | `ms` | Measures inbound HTTP request duration in `Federation.fetch()`. |

### Metric attributes

Expand All @@ -324,11 +326,42 @@ Fedify records the following OpenTelemetry metrics:
: `activitypub.verification.failure_reason`, plus
`activitypub.remote.host` when the failed signature includes a key ID.

`fedify.http.server.request.count` and `fedify.http.server.request.duration`
: `http.request.method` and `fedify.endpoint` are always present.
`http.request.method` is normalized to one of the standard HTTP methods
(`CONNECT`, `DELETE`, `GET`, `HEAD`, `OPTIONS`, `PATCH`, `POST`, `PUT`,
`QUERY`, `TRACE`) or `_OTHER` for any other value, so that an arbitrary
client cannot inflate metric cardinality by sending custom methods.
`http.response.status_code` is recorded when a `Response` is produced
(success and non-2xx alike) and omitted when the request threw an
exception before a response could be returned. `fedify.route.template`
is recorded when a route matched, and contains the [URI Template]
parameter names (for example `/users/{identifier}`) rather than the
matched parameter values.

Fedify records `activitypub.remote.host` as the URL hostname only; ports, paths,
and query strings are deliberately excluded to keep metric cardinality bounded.
Activity types use the same qualified URI form as Fedify's trace attributes,
for example `https://www.w3.org/ns/activitystreams#Create`.

The HTTP server request metrics deliberately exclude high-cardinality fields
such as the full URL, raw path, query string, actor identifier, and inbox
URL. Use the request span's `url.full` attribute when you need the exact URL
for a sampled trace; the metrics expose the stable endpoint category and route
template so that aggregate request rate, latency, and status-code error rate
remain meaningful even when traces are sampled.

The `fedify.endpoint` attribute is drawn from a fixed enumeration:
`webfinger`, `nodeinfo`, `actor`, `inbox`, `shared_inbox`, `outbox`,
`object`, `following`, `followers`, `liked`, `featured`, `featured_tags`,
`collection`, `not_found`, `not_acceptable`, and `error`. When a request
throws an exception after Fedify has already classified its endpoint, the
metric retains the matched endpoint (for example `actor`) so that
fault-attribution stays per endpoint; `error` is only used when classification
itself failed.

[URI Template]: https://datatracker.ietf.org/doc/html/rfc6570


Semantic [attributes] for ActivityPub
-------------------------------------
Expand Down Expand Up @@ -367,6 +400,8 @@ for ActivityPub:
| `docloader.context_url` | string | The URL of the JSON-LD context document (if provided via Link header). | `"https://www.w3.org/ns/activitystreams"` |
| `docloader.document_url` | string | The final URL of the fetched document (after following redirects). | `"https://example.com/object/1"` |
| `fedify.actor.identifier` | string | The identifier of the actor. | `"1"` |
| `fedify.endpoint` | string | The bounded endpoint category that classified an inbound HTTP request handled by `Federation.fetch()`. | `"actor"` |
| `fedify.route.template` | string | The matched URI Template, with parameter names (not values). | `"/users/{identifier}"` |
| `fedify.inbox.recipient` | string | The identifier of the inbox recipient. | `"1"` |
| `fedify.object.type` | string | The URI of the object type. | `"https://www.w3.org/ns/activitystreams#Note"` |
| `fedify.object.values.{parameter}` | string[] | The argument values of the object dispatcher. | `["1", "2"]` |
Expand Down
74 changes: 74 additions & 0 deletions packages/fedify/src/federation/metrics.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ class FederationMetrics {
readonly signatureVerificationFailure: Counter;
readonly deliveryDuration: Histogram;
readonly inboxProcessingDuration: Histogram;
readonly httpServerRequestCount: Counter;
readonly httpServerRequestDuration: Histogram;

constructor(meterProvider: MeterProvider) {
const meter = meterProvider.getMeter(metadata.name, metadata.version);
Expand Down Expand Up @@ -48,6 +50,40 @@ class FederationMetrics {
unit: "ms",
},
);
this.httpServerRequestCount = meter.createCounter(
"fedify.http.server.request.count",
{
description: "HTTP requests handled by Federation.fetch().",
unit: "{request}",
},
);
this.httpServerRequestDuration = meter.createHistogram(
"fedify.http.server.request.duration",
{
description: "Duration of HTTP requests handled by Federation.fetch().",
unit: "ms",
advice: {
// Mirror the OpenTelemetry HTTP server semantic-conventions
// recommended buckets, expressed in milliseconds.
explicitBucketBoundaries: [
5,
10,
25,
50,
75,
100,
250,
500,
750,
1000,
2500,
5000,
7500,
10000,
],
},
},
);
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}

recordDelivery(
Expand Down Expand Up @@ -95,6 +131,44 @@ class FederationMetrics {
"activitypub.activity.type": activityType,
});
}

recordHttpServerRequest(
method: string,
endpoint: string,
durationMs: number,
options: { statusCode?: number; routeTemplate?: string } = {},
): void {
const attributes: Attributes = {
"http.request.method": normalizeHttpMethod(method),
"fedify.endpoint": endpoint,
};
if (options.statusCode != null) {
attributes["http.response.status_code"] = options.statusCode;
}
if (options.routeTemplate != null) {
attributes["fedify.route.template"] = options.routeTemplate;
}
this.httpServerRequestCount.add(1, attributes);
this.httpServerRequestDuration.record(durationMs, attributes);
}
}

const KNOWN_HTTP_METHODS: ReadonlySet<string> = new Set([
Comment thread
dahlia marked this conversation as resolved.
"CONNECT",
"DELETE",
"GET",
"HEAD",
"OPTIONS",
"PATCH",
"POST",
"PUT",
"QUERY",
"TRACE",
]);

function normalizeHttpMethod(method: string): string {
const upper = method.toUpperCase();
return KNOWN_HTTP_METHODS.has(upper) ? upper : "_OTHER";
}

const federationMetrics = new WeakMap<MeterProvider, FederationMetrics>();
Expand Down
Loading