Skip to content

Detect VS Code Copilot terminal via COPILOT_AGENT in LLM telemetry detection#54358

Open
Copilot wants to merge 2 commits into
mainfrom
copilot/detect-copilot-terminal
Open

Detect VS Code Copilot terminal via COPILOT_AGENT in LLM telemetry detection#54358
Copilot wants to merge 2 commits into
mainfrom
copilot/detect-copilot-terminal

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 18, 2026

The LLM environment detector was not identifying Copilot terminal sessions in VS Code. VS Code now exposes COPILOT_AGENT, so Copilot usage could be missed in telemetry without this update.

  • LLM telemetry detection

    • Extended the copilot rule in LLMEnvironmentDetectorForTelemetry to treat COPILOT_AGENT as a Copilot signal, alongside existing Copilot env vars.
    • Updated the inline rule comment to reflect the new detection surface.
  • Targeted test coverage

    • Added a telemetry test case asserting COPILOT_AGENT=1 resolves to "copilot".
new EnvironmentDetectionRuleWithResult<string>("copilot", new AnyMatchEnvironmentRule(
    new BooleanEnvironmentRule("GITHUB_COPILOT_CLI_MODE"),
    new AnyPresentEnvironmentRule("GH_COPILOT_WORKING_DIRECTORY", "COPILOT_CLI", "COPILOT_AGENT")))

Agent-Logs-Url: https://github.com/dotnet/sdk/sessions/e2f05d30-d845-49b1-b504-60ae3ed82c32

Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix LLM env detection for Copilot terminal in VSCode Detect VS Code Copilot terminal via COPILOT_AGENT in LLM telemetry detection May 18, 2026
Copilot AI requested a review from baronfel May 18, 2026 14:15
@baronfel baronfel marked this pull request as ready for review May 18, 2026 14:30
Copilot AI review requested due to automatic review settings May 18, 2026 14:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the SDK’s LLM telemetry environment detection so VS Code Copilot terminal sessions are recognized via the newly exposed COPILOT_AGENT environment variable, preventing Copilot usage from being missed in telemetry.

Changes:

  • Extended the Copilot detection rule in LLMEnvironmentDetectorForTelemetry to treat COPILOT_AGENT as a Copilot signal.
  • Updated the inline comment documenting Copilot detection inputs.
  • Added a targeted telemetry test case verifying COPILOT_AGENT=1 resolves to "copilot".

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/Cli/dotnet/Telemetry/LLMEnvironmentDetectorForTelemetry.cs Adds COPILOT_AGENT to the Copilot environment detection rule and updates the rule comment accordingly.
test/dotnet.Tests/TelemetryTests/TelemetryCommonPropertiesTests.cs Adds a new theory data case asserting COPILOT_AGENT triggers "copilot" detection.

new EnvironmentDetectionRuleWithResult<string>("copilot", new AnyMatchEnvironmentRule(
new BooleanEnvironmentRule("GITHUB_COPILOT_CLI_MODE"),
new AnyPresentEnvironmentRule("GH_COPILOT_WORKING_DIRECTORY", "COPILOT_CLI"))),
new AnyPresentEnvironmentRule("GH_COPILOT_WORKING_DIRECTORY", "COPILOT_CLI", "COPILOT_AGENT"))),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the new env variable be a boolean rule?

https://github.com/microsoft/vscode/pull/316267/changes seems to be setting it to '1'.

@baronfel baronfel enabled auto-merge (squash) May 18, 2026 14:43
{ new Dictionary<string, string> { { "GITHUB_COPILOT_CLI_MODE", "true" } }, "copilot" },
{ new Dictionary<string, string> { { "GH_COPILOT_WORKING_DIRECTORY", "/repo" } }, "copilot" },
{ new Dictionary<string, string> { { "COPILOT_CLI", "1" } }, "copilot" },
{ new Dictionary<string, string> { { "COPILOT_AGENT", "1" } }, "copilot" },
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@baronfel do we want to differentiate between the CLI and VSCode usage in our telemetry?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this field IMO no - it should attribute to copilot. For your other PR we can attribute to the 'host' (vscode, etc) and I think we should update your PR to check for this signal too.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you thinking we'll take the other PR? That one feels more hacky and harder to justify in servicing. Hence my thought of splitting this up with the simpler env check.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was thinking we'd polish it up and take it. Harder to justify in servicing, sure - but fine for the 11 and 10.0.4xx releases.

@dotnet-policy-service
Copy link
Copy Markdown
Contributor

This PR has been labeled with Area-Telemetry. Please ensure that any telemetry changes in this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LLM env detection should detect copilot terminal in VSCode

5 participants