feat: OpenTelemetry instrumentation + Agent365 baggage support#475
Open
rido-min wants to merge 28 commits into
Open
feat: OpenTelemetry instrumentation + Agent365 baggage support#475rido-min wants to merge 28 commits into
rido-min wants to merge 28 commits into
Conversation
Emits per-turn ActivitySource spans (turn / middleware / handler / auth.outbound / conversation_client) and Meter instruments (teams.activities.received, teams.turn.duration, teams.handler.errors, teams.middleware.duration, teams.outbound.calls, teams.outbound.errors) from the Teams SDK pipeline so consuming bots can wire telemetry through the Microsoft OpenTelemetry distro. Layering: Core publishes "Microsoft.Teams.Core" source/meter (turn / middleware / auth.outbound / conversation_client); Apps publishes "Microsoft.Teams.Apps" source for the handler span. Each layer owns its own ActivitySource without cross-references. Adds two layer-specific BaggageBuilder classes for Agent365 export (same name in different namespaces, no inheritance): Core's reads from CoreActivity / ConversationAccount with channelData.tenant.id JSON fallback; Apps's adds the keys backed by TeamsConversationAccount (user.id, user.email, microsoft.agent.user.email, gen_ai.agent.description) and uses typed TeamsChannelData for the tenant fallback. Promotes TenantId to a typed property on Core's ConversationAccount so core/observability stays free of Apps-layer dependencies for cert attribute coverage. Adds the design doc at core/docs/Observability-Design.md (layering constraints, span/metric maps, crisp Agent365 cert definition with per-scope attribute tables, channel/sub-channel resolution) and a runnable sample at core/samples/ObservabilityBot/ wiring UseMicrosoftOpenTelemetry with OTLP export. Picks up one new package dep on Microsoft.Teams.Core: OpenTelemetry.Api 1.15.3 (lightweight contract package needed for OpenTelemetry.Baggage.Current). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces TeamsCoreTelemetry with CoreTelemetryNames for ActivitySource and Meter names. Integrates OpenTelemetry tracing/metrics and Azure OpenAI chat client in Program.cs. Adds ModelContextProtocol tools and citation processing. Updates dependencies and unit tests to use new telemetry naming.
Updated target framework to net10.0 and upgraded Microsoft.OpenTelemetry to 1.0.2. Enabled S2S endpoint for Agent365 exporter and adjusted TokenResolver to use the correct API scope and return the token without the "Bearer" prefix.
Introduce InvokeAgentScope to emit Agent365-compatible OpenTelemetry spans for each bot turn, including required tags and message schema. Move BaggageBuilder usage to TeamsBotApplication for consistent baggage propagation. Remove redundant baggage setup in Program.cs. Minor formatting update in AppsTelemetry.cs. Improves observability and telemetry partitioning for Agent365.
Moved Teams bot message handling and chat logic from Program.cs into a new ObservabilityBotApp class for better separation of concerns. Updated dependency injection for OpenTelemetry and MCP client. Modified core.slnx to disable building certain projects by default in Debug configuration.
Cleaned up core.slnx and ObservabilityBot.csproj by removing the project reference to the local Microsoft.OpenTelemetry and adding a direct NuGet package reference instead. In Program.cs, removed registration and usage of "calendarTools" and "teams" McpClient services, updating ChatOptions to use only "msdocs" tools and instructions.
… TeamsConversationAccount.FromConversationAccount - Router.DispatchAsync/DispatchWithReturnAsync: replace span?.AddException(ex) with span.RecordException(ex). The BCL Activity.AddException only adds an exception event; it does not set ActivityStatusCode.Error, leaving handler failures unflagged in spans. RecordException is the project-local extension already used by every other catch in the SDK and matches the documented contract. - TeamsConversationAccount.FromConversationAccount: copy the new typed TenantId from the source ConversationAccount. After TenantId was promoted from a Properties-backed accessor to a typed [JsonPropertyName(`tenantId`)] property on ConversationAccount, the conversion (used by TeamsActivity ctor and From/Recipient lazy getters) silently dropped the tenant id, breaking BaggageBuilder.FromTeamsContext's microsoft.tenant.id population. - Diagnostics.ActivityExtensions: promote from internal to public so the Apps-layer Router can use RecordException without InternalsVisibleTo. - Add regression test FromConversationAccount_PreservesTenantId. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace TeamsCoreTelemetry references with the actual public class name CoreTelemetryNames (10+ doc references plus a stale XMLDoc example in TeamsBotApplicationTelemetry.cs that would not have compiled if used as written). - Drop the `no new package dependencies` overview claim; Core takes a single new dep on OpenTelemetry.Api so BaggageBuilder can write to OpenTelemetry.Baggage.Current (already documented under Dependency impact). - auth.outbound span tags: drop auth.scope (commented out as TODO in BotAuthenticationHandler.cs:84) and drop client_credentials from auth.flow values; only agentic / app_only / managed_identity are emitted. - Rewrite the exception-recording paragraph: RecordException always uses manual event tagging on both net8.0 and net10.0; the SDK never delegates to the BCL Activity.AddException because that API does not set ActivityStatusCode.Error. Note that ActivityExtensions is now public so the Apps Router can use it. - Required baggage map: drop the AadObjectId fallback claim for microsoft.agent.user.id; both BaggageBuilder.FromCoreActivity and FromTeamsContext only read recipient.AgenticUserId. - Rename test method TeamsCoreTelemetry_ConstantsHaveExpectedValues to CoreTelemetryNames_ConstantsHaveExpectedValues to match the class under test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…meter Wires four instruments into the previously-empty `Microsoft.Teams.Apps` meter, observed by `Router.DispatchAsync` and `DispatchWithReturnAsync`: - `teams.handler.dispatched` (Counter) — one per matched-route invocation; tags: `handler.type`, `handler.dispatch`. - `teams.handler.duration` (Histogram, ms) — recorded in `finally` so exceptions are still observed; same tags. - `teams.handler.failures` (Counter) — bumped in the catch block. - `teams.handler.unmatched` (Counter) — bumped when no selector matches; tagged with `activity.type` (DispatchAsync) or `activity.type` + `invoke.name` (DispatchWithReturnAsync 501 branch). Adds matching tag/metric-name constants on `AppsTelemetry` and a `MetricCapture` test harness in `RouterTelemetryTests` (mirrors the existing pattern in `Microsoft.Teams.Core.UnitTests/Diagnostics/TelemetryTests`). Updates `Observability-Design.md` to document the new instruments and splits the metrics table by meter. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds OpenTelemetry instrumentation across the Core and Apps bot pipeline and introduces Agent365 baggage support, with docs, tests, and a new observability-focused sample.
Changes:
- Adds Core/App
ActivitySource/Meternames, spans, metrics, and exception tagging across turn, middleware, handler, auth, and conversation-client flows. - Adds Core and Apps
BaggageBuilderimplementations plusTenantIdonConversationAccountfor Agent365 enrichment. - Adds observability documentation, unit tests, and a runnable
ObservabilityBotsample.
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| core/test/Microsoft.Teams.Core.UnitTests/Diagnostics/TelemetryTests.cs | Adds Core telemetry span/metric tests. |
| core/test/Microsoft.Teams.Core.UnitTests/Diagnostics/BaggageBuilderTests.cs | Adds Core baggage builder tests. |
| core/test/Microsoft.Teams.Core.UnitTests/AssemblyInfo.cs | Disables Core unit test parallelization. |
| core/test/Microsoft.Teams.Apps.UnitTests/TeamsActivityBuilderTests.cs | Tests tenant/agentic account preservation. |
| core/test/Microsoft.Teams.Apps.UnitTests/RouterTelemetryTests.cs | Adds Apps router telemetry tests. |
| core/test/Microsoft.Teams.Apps.UnitTests/Diagnostics/BaggageBuilderTests.cs | Adds Apps baggage builder tests. |
| core/test/Microsoft.Teams.Apps.UnitTests/AssemblyInfo.cs | Disables Apps unit test parallelization. |
| core/src/Microsoft.Teams.Core/TurnMiddleware.cs | Instruments middleware execution. |
| core/src/Microsoft.Teams.Core/Schema/ConversationAccount.cs | Adds typed TenantId. |
| core/src/Microsoft.Teams.Core/Microsoft.Teams.Core.csproj | Adds OpenTelemetry.Api. |
| core/src/Microsoft.Teams.Core/Hosting/BotAuthenticationHandler.cs | Adds outbound auth span tagging. |
| core/src/Microsoft.Teams.Core/Diagnostics/Telemetry.cs | Adds Core telemetry singletons/instruments. |
| core/src/Microsoft.Teams.Core/Diagnostics/InvokeAgentScope.cs | Adds internal Agent365 invoke scope. |
| core/src/Microsoft.Teams.Core/Diagnostics/CoreTelemetryNames.cs | Adds public Core source/meter names. |
| core/src/Microsoft.Teams.Core/Diagnostics/BaggageBuilder.cs | Adds Core Agent365 baggage builder. |
| core/src/Microsoft.Teams.Core/Diagnostics/AgentObservabilityKeys.cs | Adds Core Agent365 key constants. |
| core/src/Microsoft.Teams.Core/Diagnostics/ActivityExtensions.cs | Adds exception recording helper. |
| core/src/Microsoft.Teams.Core/ConversationClient.cs | Instruments outbound conversation operations. |
| core/src/Microsoft.Teams.Core/BotApplication.cs | Instruments turn processing and invoke scope. |
| core/src/Microsoft.Teams.Apps/TeamsBotApplication.cs | Applies Apps baggage during dispatch. |
| core/src/Microsoft.Teams.Apps/Schema/TeamsConversationAccount.cs | Uses inherited tenant property. |
| core/src/Microsoft.Teams.Apps/Routing/Router.cs | Instruments route dispatch. |
| core/src/Microsoft.Teams.Apps/GlobalSuppressions.cs | Adds naming analyzer suppression. |
| core/src/Microsoft.Teams.Apps/Diagnostics/TeamsBotApplicationTelemetry.cs | Adds public Apps source/meter names. |
| core/src/Microsoft.Teams.Apps/Diagnostics/BaggageBuilder.cs | Adds Apps Agent365 baggage builder. |
| core/src/Microsoft.Teams.Apps/Diagnostics/AppsTelemetry.cs | Adds Apps telemetry singletons/instruments. |
| core/src/Microsoft.Teams.Apps/Diagnostics/AgentObservabilityKeys.cs | Adds Apps Agent365 key constants. |
| core/samples/PABot/PABot.csproj | Updates Graph/Kiota package versions. |
| core/samples/ObservabilityBot/README.md | Documents running the observability sample. |
| core/samples/ObservabilityBot/Program.cs | Configures OpenTelemetry and app services. |
| core/samples/ObservabilityBot/ObservabilityBotApp.cs | Implements sample AI bot with Agent365 scopes. |
| core/samples/ObservabilityBot/ObservabilityBot.csproj | Adds sample project/package references. |
| core/samples/ObservabilityBot/appsettings.json | Adds sample logging settings. |
| core/docs/Observability-Design.md | Adds observability design documentation. |
| core/core.slnx | Adds ObservabilityBot to the solution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Fix named options regression in BotAuthenticationHandler (use Get(optionsName) instead of CurrentValue) - Move InvokeAgentScope inside baggage scope in TeamsBotApplication so span inherits Apps-only baggage - Record output messages on InvokeAgentScope for Agent365 certification completeness - Restore null-safe callback invocation in TurnMiddleware - Copy TenantId in CoreActivity.CloneConversationAccount - Fix activity source name mismatch in ObservabilityBot sample - Add Core telemetry sources to ObservabilityBot README example - Rename BaggageBuilder → CoreBaggageBuilder / TeamsBaggageBuilder to avoid namespace collisions - Make InvokeAgentScope public, removing need for InternalsVisibleTo Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove unused async from UseMicrosoftOpenTelemetry lambda (CS1998) - Serialize invoke response body as JSON for RecordOutputMessages - Add required Azure OpenAI env vars to README run instructions - Fix trace tree source names (Core spans → Microsoft.Teams.Core) - Fix cross-references in CoreBaggageBuilder/TeamsBaggageBuilder XML docs - Remove unused System.Reflection using from Telemetry.cs - Revert unrelated PABot dependency bump (Graph/Kiota versions) - Fix citation marker off-by-one (start loop at 0, append i+1) - Clamp CitationAppearance.Abstract to 160-char limit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Member
Author
|
@copilot check the last commit and resolve the comments that have been addressed |
Contributor
Checked the latest commit |
Member
Author
|
@copilot resolve all outdated comments and do a final review, only comment on new critical issues |
Agent-Logs-Url: https://github.com/microsoft/teams.net/sessions/1bf29a76-58a6-4f2f-b1df-30d3bd241ffb Co-authored-by: rido-min <14916339+rido-min@users.noreply.github.com>
Contributor
Done in |
MehakBindra
approved these changes
May 21, 2026
Switched s_version initialization from reflection-based runtime assembly version retrieval to using the compile-time constant ThisAssembly.NuGetPackageVersion for improved reliability and performance.
…om/microsoft/teams.net into feature/observability-otel-baggage
…-otel-baggage # Conflicts: # core/src/Microsoft.Teams.Apps/Schema/TeamsConversationAccount.cs
Add `new` keyword to TeamsConversationAccount.TenantId to fix CS0108 (hides inherited ConversationAccount.TenantId). Property proxies to base so both static and virtual dispatch see the same value. Also fall back to source.TenantId when Properties doesn't contain the key. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment on lines
+9
to
+13
| /// <summary> | ||
| /// Singletons for the SDK's <see cref="ActivitySource"/>, <see cref="Meter"/>, and instruments. | ||
| /// Internal to <c>Microsoft.Teams.Core</c>; visible to <c>Microsoft.Teams.Apps</c> | ||
| /// and <c>Microsoft.Teams.Apps.BotBuilder</c> via <c>InternalsVisibleTo</c>. | ||
| /// </summary> |
| ``` | ||
|
|
||
| The matching internal singletons live in each assembly's `Diagnostics/` folder: | ||
| - `Microsoft.Teams.Core/Diagnostics/Telemetry.cs` — owned by Core; visible to Apps and BotBuilder via `InternalsVisibleTo` (used by Core types only — Apps does not call into it). |
Comment on lines
+6
to
+8
| // Tests in this assembly use process-global state (System.Diagnostics.ActivitySource listeners, | ||
| // OpenTelemetry.Baggage.Current). Running them in parallel causes captures from one test to | ||
| // observe spans/metrics started in another. Disabling parallelization keeps the captures isolated. |
Comment on lines
+117
to
+122
| // Tenant fallback: parse channelData.tenant.id from extension data (same as Core BaggageBuilder). | ||
| if (otelActivity.GetTagItem(AgentObservabilityKeys.TenantId) is null) | ||
| { | ||
| string? channelTenantId = TryReadChannelDataTenantId(activity); | ||
| SetTagMaybe(otelActivity, AgentObservabilityKeys.TenantId, channelTenantId); | ||
| } |
Comment on lines
+135
to
+151
| try | ||
| { | ||
| var json = JsonSerializer.Deserialize<JsonElement>(frc.Result!.ToString()!); | ||
| if (json.TryGetProperty("structuredContent", out var sc) && | ||
| sc.TryGetProperty("results", out var results)) | ||
| { | ||
| return results.EnumerateArray() | ||
| .Where(r => r.TryGetProperty("contentUrl", out _)) | ||
| .Select(r => ( | ||
| Title: r.GetProperty("title").GetString() ?? "", | ||
| Url: r.GetProperty("contentUrl").GetString() ?? "", | ||
| Content: r.TryGetProperty("content", out var c) ? c.GetString() ?? "" : "" | ||
| )); | ||
| } | ||
| } | ||
| catch { } | ||
| return []; |
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <!--<PackageReference Include="Microsoft.OpenTelemetry" Version="1.0.2" />--> |
- Fix inaccurate InternalsVisibleTo claims in Telemetry.cs XML doc and Observability-Design.md (Core's Telemetry is internal to Core only) - Extract duplicated TryReadChannelDataTenantId into shared ChannelDataHelper to reduce drift risk - Mention MeterListener in Apps test AssemblyInfo parallelization comment - Narrow bare catch to catch (JsonException) in ObservabilityBot sample - Remove commented-out PackageReference in ObservabilityBot.csproj - Fix ObservabilityBot AddCitation build error after main merge (use builder API instead of extension on built TeamsActivity) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment on lines
+4
to
+9
| using System.Diagnostics; | ||
| using Microsoft.Extensions.Logging; | ||
| using Microsoft.Teams.Apps.Diagnostics; | ||
| using Microsoft.Teams.Apps.Handlers; | ||
| using Microsoft.Teams.Apps.Schema; | ||
| using Microsoft.Teams.Core.Diagnostics; |
Comment on lines
+4
to
+7
| using System.Globalization; | ||
| using System.Text.Json; | ||
| using Microsoft.Teams.Core.Schema; | ||
| using OpenTelemetry; |
Comment on lines
+215
to
+218
| | Scope | Additional Required attributes | Source | | ||
| |---|---|---| | ||
| | `InvokeAgentScope` | `gen_ai.input.messages`, `gen_ai.output.messages`, `server.address`, `server.port` | `scope.RecordInputMessages(...)` / `RecordOutputMessages(...)` + `CoreBaggageBuilder.InvokeAgentServer(host, port)` | | ||
| | `ExecuteToolScope` | `gen_ai.tool.call.arguments`, `gen_ai.tool.call.id`, `gen_ai.tool.call.result`, `gen_ai.tool.name`, `gen_ai.tool.type` | `ToolCallDetails` + `scope.RecordResponse(...)` | |
Comment on lines
+223
to
+225
|
|
||
| **Out of scope of this SDK:** the scope objects themselves (`InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope`, `OutputScope`) ship in `Microsoft.OpenTelemetry`. The Teams SDK only ships the `CoreBaggageBuilder` / `TeamsBaggageBuilder` that populates the cert-required baggage; agents create the scopes themselves at the appropriate boundaries. | ||
|
|
Comment on lines
+130
to
+138
| // Agent365: set baggage (user.id, user.email, agent details, etc.) for all | ||
| // child spans, then open the invoke_agent scope inside the baggage scope so | ||
| // the span inherits Apps-only required baggage. | ||
| using IDisposable baggageScope = new TeamsBaggageBuilder() | ||
| .FromTeamsContext(defaultContext) | ||
| .Build(); | ||
|
|
||
| using InvokeAgentScope invokeScope = InvokeAgentScope.Start(activity); | ||
|
|
Comment on lines
13
to
17
| <ItemGroup> | ||
| <PackageReference Include="Microsoft.Bot.Builder.Dialogs" Version="4.22.3" /> | ||
| <PackageReference Include="Microsoft.Kiota.Abstractions" Version="1.22.0" /> | ||
| <PackageReference Include="Microsoft.Graph" Version="5.101.0" /> | ||
| <PackageReference Include="Microsoft.Graph" Version="5.105.0" /> | ||
| <PackageReference Include="Microsoft.Kiota.Abstractions" Version="2.0.0" /> | ||
| </ItemGroup> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ActivitySourcespans (turn/middleware/handler/auth.outbound/conversation_client) andMeterinstruments (teams.activities.received,teams.turn.duration,teams.handler.errors,teams.middleware.duration,teams.outbound.calls,teams.outbound.errors) from the SDK pipeline. Each layer owns its own source: Core →Microsoft.Teams.Core, Apps →Microsoft.Teams.Apps.BaggageBuilderclasses for Agent365 export (same name in different namespaces, no inheritance). Core's reads fromCoreActivity/ConversationAccountwith achannelData.tenant.idJSON fallback; Apps's adds theTeamsConversationAccount-backed keys (user.id,user.email,microsoft.agent.user.email,gen_ai.agent.description) and uses typedTeamsChannelDatafor the tenant fallback.TenantIdto a typed property on Core'sConversationAccountso the Core layer can populatemicrosoft.tenant.idwithout Apps-layer dependencies.core/docs/Observability-Design.md(layering rules, span/metric maps, crisp Agent365 cert definition with per-scope attribute tables, Channel/SubChannel mapping resolution) and a runnable sample atcore/samples/ObservabilityBot/that wiresUseMicrosoftOpenTelemetrywith OTLP export.Microsoft.Teams.Core:OpenTelemetry.Api1.15.3 (lightweight contract package — needed forOpenTelemetry.Baggage.Current; transitive on everyMicrosoft.OpenTelemetryconsumer).Test plan
dotnet buildclean onnet8.0+net10.0forMicrosoft.Teams.Core,Microsoft.Teams.Apps,Microsoft.Teams.Apps.BotBuilder, andsamples/ObservabilityBot.dotnet test core/test/Microsoft.Teams.Core.UnitTests— 111 passing on both TFMs (96 baseline + 15 new for telemetry / baggage).dotnet test core/test/Microsoft.Teams.Apps.UnitTests— 161 passing on both TFMs (152 baseline + 9 new for handler span / baggage).dotnet test core/test/Microsoft.Teams.Apps.BotBuilder.UnitTests— 41 passing (no regressions).samples/ObservabilityBot— Microsoft.OpenTelemetry distro 1.0.1 boots cleanly, log records flow through OTel with the full resource bag.🤖 Generated with Claude Code