Use Helix in CI

.github/skills/diagnostics-pr-build-results/SKILL.md

@@ -0,0 +1,163 @@
+---
+name: diagnostics-pr-build-results
+description: Look up CI build results for a pull request to the diagnostics repository. Use this when asked to check build status, investigate CI failures, or get test results for a PR number. Covers ADO pipeline results and Helix test work items.
+---
+
+# Diagnostics PR Build Results Lookup
+
+Use this skill to find and analyze CI build results for pull requests in the dotnet/diagnostics repository.
+
+## Pipeline Details
+
+- **ADO Organization**: `dnceng-public`
+- **ADO Project**: `public`
+- **Pipeline Definition ID**: `25` (diagnostics-public-ci)
+- **Pipeline URL**: https://dev.azure.com/dnceng-public/public/_build?definitionId=25
+- **Helix API**: https://helix.dot.net/api
+
+## Step 1: Find the ADO Build for a PR
+
+### Option A: From GitHub (preferred)
+
+Use the GitHub MCP server to get check runs for the PR:
+
+1. Use `github-mcp-server-pull_request_read` with `method: "get_status"` on `owner: "dotnet"`, `repo: "diagnostics"`, `pullNumber: <PR_NUMBER>` to get the commit status and check runs.
+2. Look for the check run from Azure Pipelines — it will have a `target_url` pointing to `dev.azure.com/dnceng-public/public/_build/results?buildId=<BUILD_ID>`.
+3. Extract the `buildId` from the URL.
+
+### Option B: From ADO directly
+
+Query ADO for builds triggered by the PR branch:
+
+```
+GET https://dev.azure.com/dnceng-public/public/_apis/build/builds?definitions=25&branchName=refs/pull/<PR_NUMBER>/merge&$top=5&api-version=7.0
+```
+
+Use the ADO PAT for authentication (Basic auth with `:PAT` base64-encoded).
+
+## Step 2: Get Build Results from ADO
+
+Once you have the `buildId`:
+
+1. **Get the build timeline** to see all jobs and their status:
+   ```
+   GET https://dev.azure.com/dnceng-public/public/_apis/build/builds/<BUILD_ID>/timeline?api-version=7.0
+   ```
+
+2. **Find "Send Tests to Helix" tasks** in the timeline records:
+   - Filter for records where `name` contains "Send Tests to Helix"
+   - Check `result` (succeeded/failed) and `state` (completed)
+   - Each has a parent record chain: Stage > Phase > Job > Task
+
+3. **Get Helix Job IDs** from the task logs:
+   - Get the log URL from `record.log.url`
+   - Fetch the log content and search for `Sent Helix Job.*jobs/([a-f0-9-]+)` to extract Helix job IDs
+
+4. **Map jobs to build configurations** by traversing the parent chain in the timeline:
+   - Task → Job → Phase (has config name like `MacOS_arm64_Debug`) → Stage
+
+## Step 3: Query Helix for Test Results
+
+With the Helix job ID and access token:
+
+1. **List work items**:
+   ```
+   GET https://helix.dot.net/api/jobs/<JOB_ID>/workitems?api-version=2019-06-17&access_token=<TOKEN>
+   ```
+
+2. **Get work item details** (for failed items):
+   ```
+   GET https://helix.dot.net/api/jobs/<JOB_ID>/workitems/<WORK_ITEM_NAME>?api-version=2019-06-17&access_token=<TOKEN>
+   ```
+   Returns `State` (Failed/Passed), `ExitCode`, `ConsoleOutputUri`, `Files`.
+
+3. **Get console log** for a failed work item:
+   ```
+   GET <ConsoleOutputUri>&access_token=<TOKEN>
+   ```
+   Search for `FAIL`, `Error Message:`, `Assert`, `Exception` lines.
+
+## Step 4: Summarize Results
+
+Present results in a table format grouped by platform:
+
+```
+| Platform          | Config  | Test Suite        | Result | Details           |
+|-------------------|---------|-------------------|--------|-------------------|
+| Linux_x64         | Debug   | EventPipe         | PASS   |                   |
+| Linux_x64         | Debug   | SOS.UnitTests     | FAIL   | LLDB 14 broken    |
+```
+
+## Authentication
+
+### ADO PAT
+Use Basic authentication with the PAT. The user should provide the PAT, or it may be in the session context.
+```powershell
+$b64 = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$pat"))
+$hdrs = @{ Authorization = "Basic $b64" }
+Invoke-RestMethod -Uri $url -Headers $hdrs
+```
+
+### Helix Access Token
+Append `&access_token=<TOKEN>` to Helix API URLs. The user should provide the token, or it may be in the session context.
+
+## Common Test Suites
+
+These are the Helix work items sent by the diagnostics pipeline:
+
+| Work Item | Description |
+|-----------|-------------|
+| EventPipe.UnitTests | EventPipe functionality |
+| Microsoft.Diagnostics.Monitoring.UnitTests | Monitoring library |
+| Microsoft.FileFormats.UnitTests | File format parsing |
+| Microsoft.SymbolStore.UnitTests | Symbol store |
+| DotnetCounters.UnitTests | dotnet-counters tool |
+| DotnetTrace.UnitTests | dotnet-trace tool |
+| Microsoft.Diagnostics.NETCore.Client.UnitTests | Diagnostics client library |
+| Microsoft.Diagnostics.Monitoring.EventPipe.UnitTests | EventPipe monitoring |
+| SOS.UnitTests | SOS debugger extension (uses LLDB/cdb) |
+| DbgShim.UnitTests | Debug shim (native dependencies) |
+
+## Build Configurations
+
+The pipeline runs these configurations (defined in `diagnostics.yml`):
+
+| OS | Architecture | Config | Helix Queue |
+|----|-------------|--------|-------------|
+| Windows_NT | x64 | Debug/Release | Windows.Amd64.Server2022.Open |
+| Windows_NT | x86 | Release | Windows.Amd64.Server2022.Open |
+| linux | x64 | Debug/Release | Ubuntu.2204.Amd64.Open |
+| linux (musl) | x64 | Debug/Release | Ubuntu.2204.Amd64.Open |
+| osx | x64 | Debug/Release | OSX.15.Amd64.Open |
+| osx | arm64 | Debug/Release | OSX.15.Arm64.Open |
+
+## Quick Reference Script
+
+Here's a PowerShell script to quickly get all Helix results for a build:
+
+```powershell
+$pat = "<ADO_PAT>"
+$helixToken = "<HELIX_TOKEN>"
+$buildId = "<BUILD_ID>"
+
+$b64 = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$pat"))
+$hdrs = @{ Authorization = "Basic $b64" }
+
+# Get timeline
+$timeline = Invoke-RestMethod -Uri "https://dev.azure.com/dnceng-public/public/_apis/build/builds/$buildId/timeline?api-version=7.0" -Headers $hdrs
+
+# Find Helix tasks and extract job IDs
+$helixTasks = $timeline.records | Where-Object { $_.name -like "*Send Tests to Helix*" -and $_.state -eq 'completed' }
+foreach ($ht in $helixTasks) {
+    $parent = $timeline.records | Where-Object { $_.id -eq $ht.parentId }
+    $grandparent = $timeline.records | Where-Object { $_.id -eq $parent.parentId }
+    $logContent = Invoke-RestMethod -Uri $ht.log.url -Headers $hdrs
+    if ($logContent -match 'jobs/([a-f0-9-]+)') {
+        $jobId = $Matches[1]
+        Write-Host "$($grandparent.name): $($ht.result) | jobId=$jobId"
+        # Get work items
+        $items = Invoke-RestMethod "https://helix.dot.net/api/jobs/$jobId/workitems?api-version=2019-06-17&access_token=$helixToken"
+        # ... process items
+    }
+}
+```

Directory.Build.props

@@ -4,15 +4,6 @@
     <PublishWindowsPdb>false</PublishWindowsPdb>
   </PropertyGroup>
 
-  <PropertyGroup>
-    <UseSharedCompilation>false</UseSharedCompilation>
-    <LangVersion>Latest</LangVersion>
-    <WarningLevel>4</WarningLevel>
-    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
-    <SuppressNETCoreSdkPreviewMessage>true</SuppressNETCoreSdkPreviewMessage>
-    <_SkipUpgradeNetAnalyzersNuGetWarning>true</_SkipUpgradeNetAnalyzersNuGetWarning>
-  </PropertyGroup>
-
   <PropertyGroup>
     <PackageIconUrl>http://go.microsoft.com/fwlink/?LinkID=288859</PackageIconUrl>
     <RepositoryType>git</RepositoryType>
@@ -21,6 +12,13 @@
     <RepositoryName>diagnostics</RepositoryName>
   </PropertyGroup>
 
+  <!--
+    Common.props contains build properties shared between local builds and Helix agents.
+    It is imported here for local builds, and separately by
+    AuxMsbuildFiles/Directory.Build.props for debuggee builds on Helix where the
+    Arcade SDK is not available.
+  -->
+  <Import Project="eng/Common.props" />
   <Import Project="Sdk.props" Sdk="Microsoft.DotNet.Arcade.Sdk" />
 
   <!--
@@ -57,21 +55,4 @@
     <TargetRid>$(TargetRidOS)-$(TargetArch)</TargetRid>
   </PropertyGroup>
 
-  <PropertyGroup>
-    <DesktopTargetFramework>net462</DesktopTargetFramework>
-    <!--
-        This represents the minimum supported .NET Version, so the min version that the tools must
-        be able to run against for a simple customer experience.
-
-        When bumping this, bump __targetFramework in build.sh/build-native.cmd and modify the
-        Debugger.Tests.Configs to reflect the new TFMs
-     -->
-    <NetCoreAppMinVersion>8.0</NetCoreAppMinVersion>
-    <NetCoreAppMinTargetFramework>net$(NetCoreAppMinVersion)</NetCoreAppMinTargetFramework>
-    <!-- This is the list of TFMs we build our debuggees and tracees as. -->
-    <SupportedSubProcessTargetFrameworks>net8.0;net9.0;net10.0</SupportedSubProcessTargetFrameworks>
-    <!-- This is the list of TFMs we build our unit tests as. -->
-    <SupportedXUnitTestTargetFrameworks>net8.0</SupportedXUnitTestTargetFrameworks>
-  </PropertyGroup>
-
 </Project>

diagnostics.yml

@@ -19,6 +19,10 @@ parameters:
   displayName: Build only (skip tests)
   type: boolean
   default: false
+- name: useHelix
+  displayName: Run tests on Helix (distributed testing)
+  type: boolean
+  default: true
 
 trigger: none
 
@@ -52,7 +56,7 @@ extends:
   parameters:
     stages:
     - stage: build
-      displayName: Build and Test Diagnostics
+      displayName: Build Diagnostics
       jobs:
 
         ############################
@@ -80,6 +84,9 @@ extends:
           name: Windows
           osGroup: Windows_NT
           buildOnly: ${{ parameters.buildOnly }}
+          useHelix: ${{ parameters.useHelix }}
+          isOfficialBuild: ${{ eq(variables['System.TeamProject'], 'internal') }}
+          helixQueues: $(HelixQueuesWindows_x64)
           buildConfigs:
           - configuration: Debug
             architecture: x64
@@ -100,7 +107,10 @@ extends:
           osGroup: Linux
           container: linux_x64
           crossBuild: true
-          buildOnly: true
+          buildOnly: ${{ parameters.buildOnly }}
+          useHelix: ${{ parameters.useHelix }}
+          isOfficialBuild: ${{ eq(variables['System.TeamProject'], 'internal') }}
+          helixQueues: $(HelixQueuesLinux_x64)
           buildConfigs:
           - configuration: Release
             architecture: x64
@@ -118,7 +128,10 @@ extends:
           osSuffix: -musl
           container: linux_musl_x64
           crossBuild: true
-          buildOnly: true
+          buildOnly: ${{ parameters.buildOnly }}
+          useHelix: ${{ parameters.useHelix }}
+          isOfficialBuild: ${{ eq(variables['System.TeamProject'], 'internal') }}
+          helixQueues: $(HelixQueuesLinux_musl_x64)
           buildConfigs:
           - configuration: Release
             architecture: x64
@@ -135,6 +148,9 @@ extends:
           jobTemplate: ${{ variables.jobTemplate }}
           osGroup: MacOS
           buildOnly: ${{ parameters.buildOnly }}
+          useHelix: ${{ parameters.useHelix }}
+          isOfficialBuild: ${{ eq(variables['System.TeamProject'], 'internal') }}
+          helixQueues: $(HelixQueuesMacOS_x64)
           buildConfigs:
           - configuration: Release
             architecture: x64
@@ -148,7 +164,10 @@ extends:
           jobTemplate: ${{ variables.jobTemplate }}
           osGroup: MacOS
           crossBuild: true
-          buildOnly: true
+          buildOnly: ${{ parameters.buildOnly }}
+          useHelix: ${{ parameters.useHelix }}
+          isOfficialBuild: ${{ eq(variables['System.TeamProject'], 'internal') }}
+          helixQueues: $(HelixQueuesMacOS_arm64)
           buildConfigs:
           - configuration: Release
             architecture: arm64
@@ -176,7 +195,10 @@ extends:
             osGroup: Linux
             container: linux_arm64
             crossBuild: true
-            buildOnly: true
+            buildOnly: ${{ parameters.buildOnly }}
+            useHelix: ${{ parameters.useHelix }}
+            isOfficialBuild: ${{ eq(variables['System.TeamProject'], 'internal') }}
+            helixQueues: $(HelixQueuesLinux_arm64)
             buildConfigs:
             - configuration: Release
               architecture: arm64
@@ -205,86 +227,20 @@ extends:
             osSuffix: -musl
             container: linux_musl_arm64
             crossBuild: true
-            buildOnly: true
+            buildOnly: ${{ parameters.buildOnly }}
+            useHelix: ${{ parameters.useHelix }}
+            isOfficialBuild: ${{ eq(variables['System.TeamProject'], 'internal') }}
+            helixQueues: $(HelixQueuesLinux_musl_arm64)
             buildConfigs:
             - configuration: Release
               architecture: arm64
               artifactUploadPath: bin/linux.arm64.Release
               artifactTargetPath: bin/linux-musl.arm64.Release
 
-      ############################
-      #                          #
-      #      Test only legs      #
-      #                          #
-      ############################
-
-      - ${{ if ne(parameters.buildOnly, true) }}:
-        - template: /eng/pipelines/build.yml
-          parameters:
-            jobTemplate: ${{ variables.jobTemplate }}
-            name: Ubuntu_22_04
-            osGroup: Linux
-            container: test_ubuntu_22_04
-            dependsOn: Linux
-            testOnly: true
-            buildConfigs:
-            - configuration: Release
-              architecture: x64
-            - ${{ if in(variables['Build.Reason'], 'PullRequest') }}:
-              - configuration: Debug
-                architecture: x64
-
-        - template: /eng/pipelines/build.yml
-          parameters:
-            jobTemplate: ${{ variables.jobTemplate }}
-            name: Alpine3_19
-            osGroup: Linux
-            osSuffix: -musl
-            container: test_linux_musl_x64
-            dependsOn: Linux_musl
-            testOnly: true
-            disableComponentGovernance: true
-            buildConfigs:
-            - configuration: Release
-              architecture: x64
-            - ${{ if in(variables['Build.Reason'], 'PullRequest') }}:
-              - configuration: Debug
-                architecture: x64
-
-        - ${{ if ne(variables['System.TeamProject'], 'public') }}:
-          - template: /eng/pipelines/build.yml
-            parameters:
-              jobTemplate: ${{ variables.jobTemplate }}
-              name: Debian_Bullseye
-              osGroup: Linux
-              container: test_debian_11_amd64
-              dependsOn: Linux
-              testOnly: true
-              buildConfigs:
-              - configuration: Release
-                architecture: x64
-              - ${{ if in(variables['Build.Reason'], 'PullRequest') }}:
-                - configuration: Debug
-                  architecture: x64
-
-          - template: /eng/pipelines/build.yml
-            parameters:
-              jobTemplate: ${{ variables.jobTemplate }}
-              name: Fedora_39
-              osGroup: Linux
-              container: test_fedora
-              dependsOn: Linux
-              testOnly: true
-              buildConfigs:
-              - configuration: Release
-                architecture: x64
-              - ${{ if in(variables['Build.Reason'], 'PullRequest') }}:
-                - configuration: Debug
-                  architecture: x64
-
     - ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
       - stage: package
         displayName: Package, Sign, and Generate BAR Manifests
+        dependsOn: build
         jobs:
         - template: /eng/common/templates-official/job/job.yml
           parameters: