Skip to content

feat(ci): add release-dry-run.yml — catch release.yml regressions at PR-time (A1)#353

Merged
BaseInfinity merged 3 commits into
mainfrom
feat/release-dry-run-ci
May 25, 2026
Merged

feat(ci): add release-dry-run.yml — catch release.yml regressions at PR-time (A1)#353
BaseInfinity merged 3 commits into
mainfrom
feat/release-dry-run-ci

Conversation

@BaseInfinity
Copy link
Copy Markdown
Owner

Summary

A1 from ROADMAP follow-up + v1.75.1 post-mortem. Runs `npm publish --dry-run` on every PR touching release.yml, package.json, or the shipped package surface. Catches MODULE_NOT_FOUND-class regressions, dropped shipped paths, npm/Node version drift — visible at PR review instead of producing a half-published tag.

What's in the box

  • `.github/workflows/release-dry-run.yml` (134 lines, heavily commented)
  • `tests/test-release-dry-run-workflow.sh` (25 quality tests)
  • `ci.yml` wiring so the test suite gates the workflow's shape

Cross-model design (Codex gpt-5.5 xhigh, .reviews/176-followup-prio-codex.md)

Codex caught 3 things I missed:

  1. OIDC mint risk — npm CLI runs OIDC setup before the dry-run branch. With `id-token: write` present, npm may attempt token mint even though it never PUTs. Workflow uses `permissions: contents: read` only. Dry-run proves packaging; publish auth stays in release.yml.
  2. Already-published collision — Naive `npm publish --dry-run` against `1.75.1` errors with "cannot publish over previously published versions" (verified locally). `--force` bypasses but disables protections. Workflow rewrites `package.json.version` to `0.0.0-dry-run-${GITHUB_SHA::7}` in a temp checkout instead.
  3. Path-drift guard — Test asserts every `package.json.files` entry is in the workflow `paths:` filter. Without it, adding a new shipped path would silently bypass the dry-run.

Test plan

  • TDD RED: 19/25 fail before workflow exists (negative checks trivially pass)
  • TDD GREEN: 25/25 pass after workflow + iteration on comment-matching grep pattern
  • No regression in `test-release-workflow.sh` (15/15)
  • No regression in `test-doc-consistency.sh` (40/40)
  • CI `validate` green on this PR
  • Self-test: the workflow itself runs on this PR (release-dry-run path matches) and succeeds against the temp-rewritten version

What this does NOT catch (honest scope)

  • The exact v1.75.0 MODULE_NOT_FOUND failure (caused by deleted `npm install -g npm@latest` step). Codex's note: this catches the REPLACEMENT contract (Node 24 + bundled npm + version guard), not the removed self-upgrade path.

…PR-time

A1 from .reviews/176-followup-prio-codex.md, originated from v1.75.1
post-mortem item (a). Runs 'npm publish --dry-run' on PRs touching
release.yml, package.json, the cli/, hooks/, skills/, .claude-plugin/
trees, or related docs. Catches the class of bugs that would break
release.yml at tag-time (npm/Node version drift, package.json.files
dropping a shipped path, etc.) — visible at PR review instead of
producing a half-published tag.

Design constraints (each enforced by tests/test-release-dry-run-workflow.sh,
25 quality tests):

- permissions: contents: read only. **NEVER** id-token: write. The npm
  CLI runs OIDC setup before the dry-run branch; with id-token: write
  present it may attempt token mint even though it never PUTs the
  package. Dry-run proves PACKAGING; publish auth stays in release.yml.
- Temp version rewrite to 0.0.0-dry-run-<SHA> before publishing.
  Naive 'npm publish --dry-run' against the checked-in 1.x.y errors
  with 'cannot publish over previously published versions'. --force
  works but disables protections; Codex flagged this.
- Node 24 + npm >=11.5.1 guard, parity-tested against release.yml.
- No --force, no --provenance, no NODE_AUTH_TOKEN, no NPM_TOKEN.
- Path filter covers every package.json.files entry; new test asserts
  this so a future entry can't silently bypass the dry-run.

Wired into ci.yml so the test suite gates the workflow's shape too.
TDD: 25/25 tests RED before workflow → 25/25 GREEN after. No regression
in existing release-workflow tests (15/15) or doc-consistency (40/40).

Per cross-model design review (.reviews/176-followup-prio-codex.md
section 2.A1 + section 3 risks 1-3, 9).
…w dry-run test in CONTRIBUTING

PR 353 CI caught two cross-cutting tests that need to know about the new
release-dry-run.yml workflow:

1. test_no_unused_id_token_permission was matching the literal string
   'id-token: write' in YAML comments. Workflows like release-dry-run.yml
   reference the permission in their explanatory comments to document WHY
   they DO NOT request it — that documentation should not trip the test.
   Anchored the grep to active code lines (same '^[[:space:]]*[^#[:space:]]'
   pattern used by test_no_node_auth_token in the new test file).

2. CONTRIBUTING.md test-script list rebuilt: added test-release-dry-run-workflow.sh
   between test-release-workflow.sh and test-domain-detection.sh.

Both PASS locally now. Original 25/25 dry-run tests still pass.
@BaseInfinity BaseInfinity force-pushed the feat/release-dry-run-ci branch from 6decf91 to 7f952fa Compare May 24, 2026 23:50
@BaseInfinity BaseInfinity enabled auto-merge (squash) May 24, 2026 23:50
@BaseInfinity BaseInfinity merged commit ef40984 into main May 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant