feat(appkit): send internal telemetry via AppkitLog schema#332
Open
calvarjorge wants to merge 20 commits intomainfrom
Open
feat(appkit): send internal telemetry via AppkitLog schema#332calvarjorge wants to merge 20 commits intomainfrom
calvarjorge wants to merge 20 commits intomainfrom
Conversation
Introduce the AppkitLog event family (APP_STARTUP, HEARTBEAT,
REQUEST_METRICS) and a TelemetryReporter singleton that owns the
shared dispatch state, periodic heartbeat, and request metrics
aggregation. The server plugin records each matched route via
res.on('finish') middleware; the reporter flushes one event per
endpoint on a periodic timer. The legacy observability_log
APP_STARTUP is kept as a fallback until the AppkitLog schema is
deployed end-to-end on the telemetry backend.
Errors propagate from the inner senders so consumers can see
exactly what was POSTed and how the endpoint responded; the only
catches live at the SDK's outermost boundaries (fire-and-forget
startup + interval timers).
Adds an Internal Telemetry tab in dev-playground that lets you
trigger each event on demand and renders the request, response,
and an equivalent curl command. Disable with
disableInternalTelemetry: true or APPKIT_TELEMETRY_DISABLED=true.
Also unblocks the pre-commit knip hook by ignoring the @cyclonedx/cdxgen
dependency, which is invoked dynamically via pnpm exec from the
release:sbom script.
Co-authored-by: Isaac
Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
The dev-playground registers the jobs() plugin, whose manifest requires DATABRICKS_JOB_ID, but app.yaml never declared a binding for it. As a result, deploying the playground to a fresh Databricks App fails AppKit's startup resource validation with "Missing required resources: job:Job [jobs]". Add the missing entry alongside the other resource bindings. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
The bare /telemetry endpoint rejects SP bearer tokens and 302s to /login.html?next_url=..., which the previous code tried to follow verbatim — but a relative location is not a valid fetch URL and threw a "Failed to parse URL" error that the legacy try/catch silently swallowed. Switch the dispatch URL to the SP-friendly /telemetry-ext endpoint, and harden redirect handling by resolving the location against the original request URL. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
Now that the AppkitLog dispatch path is verified end-to-end against /telemetry-ext, the observability_log fallback is no longer needed. Remove sendStartupTelemetry, the dead StartupTelemetryParams / buildEntityId / buildLegacyStartupPayload helpers, and the second fire-and-forget block in createApp's bootstrap. Migrate the sender test suite to cover sendAppkitLogs (which retains all the URL, auth, redirect, and error-propagation guarantees) and rewrite the core internal-telemetry tests against the reporter mock. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
Databricks Apps injects DATABRICKS_CLIENT_ID (the app's OAuth client UUID) into the runtime env, not DATABRICKS_APP_ID, so the old lookup always resolved to "" and the AppkitLog.app_id field went out empty. Switch the bootstrap to read DATABRICKS_CLIENT_ID so logs carry the actual per-app identifier. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
Rename APPKIT_TELEMETRY_DISABLED to DISABLE_APPKIT_INTERNAL_TELEMETRY.
The new name makes it explicit that this controls AppKit's
internal/anonymized telemetry, not the user-facing OpenTelemetry
config exposed via createApp({ telemetry }).
Co-authored-by: Isaac
Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
sender.ts only wrapped postTelemetry with buildAppkitPayload — one caller, no shared logic worth a dedicated file. Have the reporter's #send call postTelemetry directly and rename the test file from sender.test.ts to client.test.ts so the wire-format coverage (URL, auth, redirects, error propagation) clearly targets postTelemetry. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
Document exactly what AppKit collects (event_name, app_id, appkit_version, plus per-event bodies for APP_STARTUP, HEARTBEAT, and REQUEST_METRICS), how it's sent, and the two ways to disable — disableInternalTelemetry on createApp and the DISABLE_APPKIT_INTERNAL_TELEMETRY env var. Bump faq.md's sidebar position to 8 to make room. Add a one-paragraph header in the package's index.ts pointing at the public doc. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
- Refuse cross-origin redirects from /telemetry-ext so the live SP Authorization header cannot be replayed against a third party. - Fall back to serviceCtx.client.config.host when DATABRICKS_HOST is unset so the dispatch URL still resolves correctly when the SDK was given a pre-configured WorkspaceClient. - Redact Authorization / Cookie / Set-Cookie when surfacing the request in the dev-playground debug UI and in the printed curl, so the sensitive headers don't leak via the response or get copy-pasted into shared logs. - Revert the knip.json @cyclonedx/cdxgen exception. Earlier diagnosis was wrong — the warnings are notices, not errors, and the original pre-commit failures came from this branch's own unused exports (now trimmed). With the branch rebased on origin/ main, knip exits 0 against the unmodified config. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
Both branches in fetchWithRedirect (cross-origin throw + same-origin follow) want to release the redirect's response body before moving on. Run the cancel once after we've parsed the target URL, before deciding what to do, instead of repeating it on each side. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
The /telemetry-ext endpoint may not actually issue redirects in practice; the follow logic was added defensively. Inline a single fetch with redirect: "manual" so any 3xx surfaces directly to the caller (and the dev-playground UI), making it easy to verify whether the redirect path is exercised on real traffic. If the deployed app never produces a 3xx response, the helper function and its tests stay deleted; if it does, restore them. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
calvarjorge
force-pushed
the
jorge.calvar/send_telemetry
branch
from
04a9847 to
08378b9
Compare
The /telemetry-ext endpoint does not actually redirect under normal SP-authenticated traffic, so the redirect-follow logic, the redirect: "manual" hint on fetch, and the corresponding tests were all dead weight. Inline a plain fetch and remove the related test. Also drop the "Inspecting events locally" section from the public internal-telemetry doc — the dev-playground harness is internal tooling and isn't relevant to library consumers. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
pkosiec
reviewed
May 4, 2026
Member
pkosiec
left a comment
pkosiec
left a comment
There was a problem hiding this comment.
Nice work! Could you share internally a short demo how it works E2E? not sure how to test it properly.
Thanks!
| cache?: CacheConfig; | ||
| client?: WorkspaceClient; | ||
| onPluginsReady?: (appkit: PluginMap<T>) => void | Promise<void>; | ||
| disableInternalTelemetry?: boolean; |
Member
There was a problem hiding this comment.
I'd vote for removing that. IMO the environmental variable is enough 👍
Contributor
Author
There was a problem hiding this comment.
This was the proposal in the design doc. Since we're obtaining some data from customer apps, we wanted to be as transparent as possible about it.
| ``` | ||
|
|
||
| Either one fully disables the reporter — no events are emitted and no | ||
| network calls are made. |
Member
There was a problem hiding this comment.
We could also support DO_NOT_TRACK=1 - https://donottrack.sh/
Contributor
Author
There was a problem hiding this comment.
The page is not loading for me
Contributor
Author
There was a problem hiding this comment.
Added DO_NOT_TRACK=1 as a way to disable it nevertheless
MarioCadenas
reviewed
May 4, 2026
| appkitVersion: productVersion, | ||
| }); | ||
| reporter.start(); | ||
| reporter.sendStartup().catch(() => {}); |
Collaborator
There was a problem hiding this comment.
we are calling this (which is asynchronous) and basically ignoring the promise, is this on purpose?
Contributor
Author
There was a problem hiding this comment.
Yes, we don't want to cause delays or raise errors because of internal telemetry.
… debug The dev-playground debug plugin was the only consumer of TelemetryReporter, TelemetrySendRequest, TelemetrySendResponse, and TelemetrySendResult outside the package. Since dev-playground is internal tooling for AppKit maintainers, exporting infrastructure types via the public surface only to feed it isn't justified. Drop: - TelemetryReporter and the TelemetrySend* types from the public @databricks/appkit exports - The TelemetrySend* re-exports from internal-telemetry/index.ts (no remaining consumer) - The internal-telemetry-debug plugin and its routes - The /internal-telemetry route, home-page card, and nav link in dev-playground The reporter and types remain available intra-package for the core/server wiring; nothing about the dispatch behavior changes. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
- Rename docs/docs/internal-telemetry.mdx to privacy.mdx so library
consumers don't confuse it with the user-facing OpenTelemetry
config exposed via createApp({ telemetry }).
- Move the page to the bottom of the sidebar (sidebar_position: 99).
- Honor the cross-tool DO_NOT_TRACK=1 convention
(https://consoledonottrack.com) alongside the AppKit-specific
DISABLE_APPKIT_INTERNAL_TELEMETRY env var.
- Document the new opt-out path on the Privacy page and update the
in-source comment that pointed at the old doc location.
Co-authored-by: Isaac
Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
The project uses moduleResolution: "bundler", which doesn't require explicit .js extensions on relative imports. The rest of appkit imports without them; only internal-telemetry/ added them, by mistake on my part. Strip them to match the surrounding codebase. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
Drop the "we do not send X, Y, Z" paragraph — committing to a non-collection list ties our hands if the schema later changes. Also drop the "How it's sent" section; the dispatch endpoint and fire-and-forget semantics are implementation details users don't need to track. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
The dev-playground no longer needs the aggregated plugin manifest; the file is generated by `appkit plugin sync` and isn't read by any current dev-playground flow. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
Drop the placeholder: true field from app_startup_event and heartbeat_event payloads; an empty object is enough to identify the oneof variant on the wire. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
The SDK's apiClient.request already does host resolution, authentication, query string handling, and User-Agent setting that client.ts was reimplementing by hand. Drop client.ts (and its tests), inline a single apiClient.request call inside reporter's #send, and remove the now-unused workspaceHost from the reporter constructor. Tests for the reporter switch to mocking apiClient.request on the WorkspaceClient mock instead of stubbing global fetch. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
…dleware - Call TelemetryReporter.getInstance()?.stop() in the server plugin's _gracefulShutdown so heartbeat/metric timers don't keep firing during the 15s shutdown grace window. - Export requestMetricsMiddleware as @internal and add unit tests for the integration point: matched routes, baseUrl + route template assembly, no-op when req.route is unset, no-op when the reporter is uninitialized, and 4xx/5xx status pass-through. Co-authored-by: Isaac Signed-off-by: Jorge Calvar <jorge.calvar@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AppkitLogevents (APP_STARTUP,HEARTBEAT,REQUEST_METRICS) and aTelemetryReportersingleton that owns shared dispatch state, the periodic heartbeat, and per-endpoint request metrics aggregation.res.on('finish')middleware; the reporter flushes one event per(method, route)on a periodic timer (configurable viaAPPKIT_TELEMETRY_HEARTBEAT_INTERVAL_MS/APPKIT_TELEMETRY_METRICS_FLUSH_INTERVAL_MS)./telemetry-ext?o=<workspaceId>with the workspace SP bearer token. Errors propagate from the inner senders so consumers can see exactly what was sent and how the endpoint responded; the only swallows live at the SDK's outermost boundaries (fire-and-forget startup + interval timers).curlcommand. Disable globally withdisableInternalTelemetry: trueoncreateApporAPPKIT_TELEMETRY_DISABLED=true.app.yamlwas missing theDATABRICKS_JOB_IDbinding the jobs plugin requires (deploys to a fresh app failed startup validation); knip false-positive on thepnpm exec cdxgen-invoked@cyclonedx/cdxgendep blocked all pre-commit hooks.