ci: Auto-accept API verifier changes in PRs#5228
Conversation
Adds a new `verify api` workflow that runs the ApiApprovalTests on macOS (covers .NET / netstandard / iOS / MacCatalyst / Android TFMs) and Windows (covers the net48 / .NET Framework TFM), then runs scripts/accept-verifier-changes.ps1 over the resulting *.received.* files and pushes the accepted snapshots back to the PR branch. When API changes are accepted, the PR is also labelled `public API`. For PRs from forks (where we can't push back), the workflow fails with a hint telling the contributor how to accept the changes locally. Closes #5157 Co-Authored-By: Claude <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5228 +/- ##
==========================================
- Coverage 74.13% 74.12% -0.02%
==========================================
Files 508 508
Lines 18282 18282
Branches 3574 3574
==========================================
- Hits 13554 13552 -2
- Misses 3859 3860 +1
- Partials 869 870 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit dc60272. Configure here.
| # here and rely on the produced *.received.txt files to detect and accept the change. | ||
| - name: Run API Approval Tests | ||
| continue-on-error: true | ||
| run: dotnet test ${{ matrix.slnf }} -c Release --no-build --nologo --filter "FullyQualifiedName~ApiApprovalTests" |
There was a problem hiding this comment.
API test failures ignored
Medium Severity
The API approval test step uses continue-on-error, so run-api-tests still succeeds when dotnet test fails without producing *.received.txt. No later step checks that outcome, so the workflow can pass while ApiApprovalTests actually failed for non-snapshot reasons.
Reviewed by Cursor Bugbot for commit dc60272. Configure here.
There was a problem hiding this comment.
That's fine. If that happens we'll know about it from the build workflow that is running concurrently. This workflow isn't intended to replace the full build - it's just a convenience so devs don't have to build the solution and run it on multiple operating systems every time one of the verify API files changes.
We're not actually running the tests to see if they pass or fail here - we run them simply as a way to get the latest verify files.
| - name: Download Received API Files | ||
| uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 | ||
| with: | ||
| pattern: api-verify-received-* | ||
| merge-multiple: true | ||
|
|
There was a problem hiding this comment.
Resolved in b011745
Bug: The accept-api-changes job may fail on PRs with no API changes because the download-artifact step errors when no API change artifacts are found to download.
Severity: MEDIUM
Suggested Fix
Add continue-on-error: true to the Download Received API Files step in the accept-api-changes job. This will prevent the workflow from failing when no API change artifacts are found. Apply the same fix to the equivalent step in the report-fork-api-changes job.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: .github/workflows/verify-api.yml#L94-L99
Potential issue: The `accept-api-changes` job includes a step to upload API verification
files, which uses `if-no-files-found: ignore`. Consequently, for pull requests without
any public API changes, no `api-verify-received-*` artifact is created. A subsequent
step attempts to download artifacts using the pattern `api-verify-received-*`. Since
`actions/download-artifact@v8.0.1` is expected to fail when a pattern matches no
artifacts and there is no `continue-on-error` configured, this job will likely fail on
every clean PR. The `report-fork-api-changes` job has the same vulnerability.
Also affects:
.github/workflows/verify-api.yml:148~151
Did we get this right? 👍 / 👎 to inform future reviews.
There was a problem hiding this comment.
Reverted per #5228 (comment)
Looking at the source for the action it only throws an error if there are no files to download if either isDownloadByIds or isSingleArtifactDownload is true... so we don't need continue on error in the download step.
So the initial review comment was erroneous.
The workflow-level `contents: write` token was inherited by `run-api-tests`, which builds and runs untrusted PR code. Scope tokens per-job instead: - run-api-tests: contents: read - accept-api-changes: contents: write, pull-requests: write - report-fork-api-changes: contents: read Co-Authored-By: Claude <noreply@anthropic.com>
The pattern branch of actions/download-artifact (no `name`/`artifact-ids`, only `pattern`) tolerates zero matches without erroring — see https://github.com/actions/download-artifact/blob/main/src/download-artifact.ts. So `continue-on-error: true` isn't needed for the clean-PR case and would otherwise mask genuine download failures. Co-Authored-By: Claude <noreply@anthropic.com>


Summary
Adds a new
verify apiworkflow that auto-accepts API approval test snapshot changes on PRs, similar to the existingformat-codeworkflow.ApiApprovalTestson a matrix ofmacos-15(covers .NET / netstandard / iOS / MacCatalyst / Android TFMs) andwindows-latest(covers thenet48/ .NET Framework TFM, which is the common pain point when developing on non-Windows).*.received.*files as artifacts.scripts/accept-verifier-changes.ps1, and pushes the accepted snapshots back to the PR branch.public APIformat-code.yml.The Alternative suggested in the issue (switching to
Microsoft.CodeAnalysis.PublicApiAnalyzers) is intentionally out of scope for this PR and can be considered in a follow-up.Closes #5157