feat(appkit): tracing integration for agent plugin

[hubertzub-db]

Stacked on top of #166

What this does

Adds agent tracing integration — LangChain spans flow through AppKit's global TelemetryManager with MLflow experiment headers, and trace destinations are auto-provisioned in Unity Catalog.

Changes to the core logic:

• Trace exporter accepts plugin-contributed headers (traceExporterHeaders) merged on top of base headers + auth
• NodeSDK instrumentations are now configurable via config.instrumentations (previously always used defaults)
• Server plugin gates instrumentation registration on enableDefaultTelemetry config (default true)

Reading order

1. Types first — understand the new interfaces:

   • packages/shared/src/plugin.ts — new appendTraceHeaders convention on PluginConstructor
   • packages/appkit/src/telemetry/types.ts — traceExporterHeaders field on TelemetryConfig
   • packages/appkit/src/plugins/agent/types.ts — IAgentConfig.tracing shape (false | { experimentId, ucTableName, warehouseId })
   • packages/appkit/src/plugins/server/types.ts — enableDefaultTelemetry on ServerConfig

2. Core wiring — how headers flow from plugins to the exporter:

   • packages/appkit/src/core/appkit.ts — _createApp collects appendTraceHeaders from all plugins, passes merged headers to TelemetryManager.initialize()
   • packages/appkit/src/telemetry/telemetry-manager.ts — now async; buildTraceExporterHeaders() merges base headers + Databricks auth + plugin headers; instrumentations config passthrough

3. Agent plugin — the main feature:

   • packages/appkit/src/plugins/agent/agent.ts — appendTraceHeaders (static, sync) provides experiment/UC headers; setup() calls setupExperimentTraceLocation then conditionally instruments LangChain
   • packages/appkit/src/plugins/agent/tracing.ts — thin wrapper around @arizeai/openinference-instrumentation-langchain
   • packages/appkit/src/plugins/agent/mlflow.ts — new file — setupExperimentTraceLocation + linkExperimentToLocation (ported from app-templates)

4. Instrumentation control — how users disable HTTP/Express noise:

   • packages/appkit/src/telemetry/instrumentations.ts — changed from eager static instances to lazy factory functions (http(), express())
   • packages/appkit/src/plugins/server/index.ts — gated on enableDefaultTelemetry !== false

5. Tests — small adjustments only:

   • server.test.ts — mock updated for factory function API
   • telemetry-manager.test.ts — await on initialize(), delete process.env.DATABRICKS_HOST in setup

Key design decisions

• Plugin-contributed headers: plugins declare static appendTraceHeaders() — called before telemetry init, keeps TelemetryManager free of MLflow-specific logic
• Tracing gating: requires MLFLOW_EXPERIMENT_ID; trace location setup must succeed or instrumentation is skipped entirely
• Lazy instrumentations: HttpInstrumentation/ExpressInstrumentation are created on-demand (factory functions) to avoid module patching at import time
• Catalog/schema derived from OTEL_UC_TABLE_NAME (split on dots) — no separate OTEL_UC_CATALOG/OTEL_UC_SCHEMA env vars