Skip to content

Conversation

@dsorokin-st
Copy link

@dsorokin-st dsorokin-st commented Jan 30, 2026

Motivation and Context

Tool result messages in model diagnostics are missing the tool_call_id property, making it difficult to correlate tool results with their corresponding tool calls in observability tools.

Current behavior - tool result messages only have role and content:

{"role": "tool", "name": null, "content": "Partly Cloudy, 22°C", "tool_calls": []}

After this change - includes tool_call_id for correlation:

{"role": "tool", "name": "get_weather", "tool_call_id": "call_abc123", "content": "Partly Cloudy, 22°C", "tool_calls": []}

This aligns with the OpenAI API format and enables observability tools (OpenInference, Arize, Galileo) to properly correlate tool calls with their results.

Description

Updated ToGenAIConventionsFormat in ModelDiagnostics.cs to:

  1. Check for FunctionResultContent in message items
  2. Add tool_call_id from FunctionResultContent.CallId
  3. Use FunctionResultContent.FunctionName for the name field (instead of AuthorName which is always null for tool messages)

This is a backward-compatible, additive change - existing consumers will simply ignore the new field.

Why Unit Testing Is Difficult

Unit testing this change is challenging due to ModelDiagnostics architecture:

  1. Static readonly field caching: The feature flags (s_enableDiagnostics, s_enableSensitiveEvents) are static readonly fields initialized at type load time via AppContextSwitchHelper.GetConfigValue().

  2. No test isolation: Once ModelDiagnostics is loaded by any test in the assembly, the flag values are permanently cached. Setting AppContext.SetSwitch or environment variables after type initialization has no effect.

  3. Reflection blocked: .NET prevents modification of initonly static fields after type initialization:

    System.FieldAccessException: Cannot set initonly static field 's_enableDiagnostics' after type 'ModelDiagnostics' is initialized.
    
  4. Existing precedent: The test GetInvalidResponseThrowsExceptionAndIsCapturedByDiagnosticsAsync in OpenAIChatCompletionServiceTests.cs is already skipped with [Fact(Skip = "Not working running in the console")] for the same reason.

The change has been manually verified to produce the correct output.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR title follows the format: .Net: <description>
  • I have added tests - See explanation above for why unit testing is impractical
  • I have updated the documentation (not applicable)

Tool result messages in model diagnostics are missing the tool_call_id property,
making it difficult to correlate tool results with their corresponding tool calls
in observability tools.

Changes:
- Check for FunctionResultContent in message items
- Add tool_call_id from FunctionResultContent.CallId
- Use FunctionResultContent.FunctionName for name field (instead of AuthorName
  which is always null for tool messages)

This aligns with the OpenAI API format and enables observability tools
(OpenInference, Arize, Galileo) to properly correlate tool calls with results.

This is a backward-compatible, additive change.
@moonbox3 moonbox3 added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel labels Jan 30, 2026
@dsorokin-st dsorokin-st force-pushed the fix-tool-call-id-diagnostics branch from a51cb32 to 34b5daa Compare January 30, 2026 10:31
@moonbox3 moonbox3 added kernel Issues or pull requests impacting the core kernel and removed kernel Issues or pull requests impacting the core kernel labels Jan 30, 2026
@dsorokin-st dsorokin-st force-pushed the fix-tool-call-id-diagnostics branch from 16c3f04 to 34b5daa Compare January 30, 2026 10:41
@moonbox3 moonbox3 removed the kernel Issues or pull requests impacting the core kernel label Jan 30, 2026
@dsorokin-st
Copy link
Author

@microsoft-github-policy-service agree company="ServiceTitan"

@dsorokin-st dsorokin-st marked this pull request as ready for review February 3, 2026 07:56
@dsorokin-st dsorokin-st requested a review from a team as a code owner February 3, 2026 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Issue or Pull requests regarding .NET code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants