[9.0] SDK Diff tests failing — az devops extension pip install failure

[ellahathaway]

Summary

The source-build-sdk-diff-tests pipeline (definition 1231) is failing on internal/release/9.0.1xx due to a pip failure when trying to install the azure-devops Azure CLI extension. All 6 legs (Fedora43, Ubuntu2404Arm64, AlmaLinux8, CentOSStream9, Alpine323, Ubuntu2404) fail identically in the "Find associated builds" step.

Build Link

https://dev.azure.com/dnceng/internal/_build/results?buildId=2964554&view=results

Error Output

WARNING: The command requires the extension azure-devops. It will be installed first.
ERROR: An error occurred. Pip failed with status code 1. Use --debug for more information.
Could not find a completed dotnet-dotnet build for branch 'refs/heads/internal/release/9.0.1xx'

Analysis

The "Find associated builds" step uses the az CLI to query for a completed upstream dotnet-dotnet build. The azure-devops extension is not pre-installed on the agent, and the dynamic install via pip fails with exit code 1. Because the extension never installs, the query never executes, and the step reports no build found.

This has been failing since at least April 30, 2026 (build 2964554). The pipeline has not had a successful run on this branch since then.

Suggested Fix

• Pre-install the azure-devops az CLI extension in the pipeline YAML or agent image, rather than relying on dynamic install.
• Alternatively, investigate why pip fails on these agents (possible network/proxy issue or Python environment mismatch).

[mthalman]

This was intended to be fixed by dotnet/sdk#54028. That change has synced to the public branch of the VMR: https://github.com/dotnet/dotnet/blob/release/9.0.1xx/eng/pipelines/templates/jobs/sdk-diff-tests.yml#L32-L34. But does not exist in the internal branch of the VMR: https://dev.azure.com/dnceng/internal/_git/dotnet-dotnet?path=/eng/pipelines/templates/jobs/sdk-diff-tests.yml&version=GBinternal/release/9.0.1xx&line=31&lineEnd=32&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents

@premun - Can you check on the state of the syncing between these two branches?

[premun]

The synchronization pipeline says it pushed just a while ago - https://dev.azure.com/dnceng/internal/_build/results?buildId=2979364&view=logs&j=f94b1352-8074-534d-f024-dae8a910f9a5&t=a1932730-9ff1-55ee-31c6-d84099cce458

But I don't see any commits on the dotnet-dotnet branch since Apr 30th.

I wonder what's going on there..

@oleksandr-didyk can you have a look at this?

[premun]

@oleksandr-didyk the order of things is:

• sdk change in release/9.0.1xx
• mirror to internal
• merge of sdk's release/9.0.1xx into internal/release/9.0.1xx
• sync pipeline from sdk to VMR

[mthalman]

Should be resolved now that release/9.0.1xx has been merged into internal/release/9.0.1xx.