Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,9 @@ jobs:
# This is needed because lint/test jobs have a transitive dependency on changeset-check
if: always() && github.ref == 'refs/heads/main' && github.event_name == 'push' && needs.lint.result == 'success' && needs.test.result == 'success' && needs.verbose-integration.result == 'success'
runs-on: ubuntu-latest
concurrency:
group: release-main
cancel-in-progress: false
# Permissions required for npm OIDC trusted publishing
permissions:
contents: write
Expand Down Expand Up @@ -256,6 +259,9 @@ jobs:
name: Instant Release
if: github.event_name == 'workflow_dispatch' && github.event.inputs.release_mode == 'instant'
runs-on: ubuntu-latest
concurrency:
group: release-main
cancel-in-progress: false
# Permissions required for npm OIDC trusted publishing
permissions:
contents: write
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,9 @@ jobs:
# Without always(), GitHub Actions skips jobs when any transitive dependency was skipped
if: always() && github.event_name == 'push' && github.ref == 'refs/heads/main' && needs.lint.result == 'success' && needs.test.result == 'success' && needs.build.result == 'success'
runs-on: ubuntu-latest
concurrency:
group: release-main
cancel-in-progress: false
permissions:
contents: write
steps:
Expand Down Expand Up @@ -281,6 +284,9 @@ jobs:
# Use always() to ensure this job runs even when changelog-check was skipped
if: always() && github.event_name == 'workflow_dispatch' && needs.lint.result == 'success' && needs.test.result == 'success' && needs.build.result == 'success'
runs-on: ubuntu-latest
concurrency:
group: release-main
cancel-in-progress: false
permissions:
contents: write
steps:
Expand Down
98 changes: 98 additions & 0 deletions docs/case-studies/issue-259/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Case Study: Issue #259 — Version Bump for Rust CI/CD Not Working

## Summary

The Rust CI/CD auto-release pipeline was failing when merging PRs to main. Two distinct root causes were identified, both related to the concurrent execution of the Rust and JS release workflows.

## Timeline of Events

| Timestamp (UTC) | Event |
|-----------------------|--------------------------------------------------------------|
| 2026-04-12 14:08 | PR #253 merged, Rust CI succeeds (v0.8.0 → v0.9.0) |
| 2026-04-13 01:55 | Push to main (cb37cb7), Rust CI succeeds (v0.9.0 released) |
| 2026-04-13 06:19 | PR #256 merge (33d1d8c), Rust CI **fails** — `CARGO_REGISTRY_TOKEN` not set |
| 2026-04-13 06:19 | JS CI runs concurrently, pushes v0.22.4 commit to main |
| 2026-04-13 06:31 | Fix commit (f675128) adds CARGO_TOKEN fallback |
| 2026-04-13 06:33 | PR CI passes on fix branch |
| 2026-04-13 06:39 | PR #258 opened with the fix |
| 2026-04-13 07:49 | PR #258 merge (53684c8), Rust CI **fails** — git push rejected (non-fast-forward) |
| 2026-04-13 07:49 | JS CI runs concurrently, pushes v0.22.5 commit to main |

## Root Causes

### Root Cause 1: Git Push Race Condition (non-fast-forward rejection)

**Symptom:** `rust-version-and-commit.mjs` fails at `git push` with:
```
! [rejected] main -> main (non-fast-forward)
error: failed to push some refs to 'https://github.com/link-assistant/agent'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart.
```

**CI Run:** [24331934514](https://github.com/link-assistant/agent/actions/runs/24331934514) (2026-04-13 07:49 UTC)

**Root Cause:** When a PR is merged to main, both the Rust and JS CI workflows trigger concurrently (because changes touch `scripts/` which is in both workflow path filters). The JS release job pushes a version commit (e.g., `0.22.5`) to main first. When the Rust release job then tries to push its version commit, it's rejected because its local `main` is behind the remote.

**Contrast with JS:** The JS `version-and-commit.mjs` already handles this by fetching `origin/main` and rebasing before committing (lines 170-207). The Rust `rust-version-and-commit.mjs` did not have equivalent logic.

**Fix:**
1. Added `git fetch origin main` + rebase before starting in `rust-version-and-commit.mjs`
2. Added push retry loop (up to 3 attempts) with `git pull --rebase` on failure
3. Added shared `concurrency: { group: release-main }` across all Rust and JS release jobs to serialize pushes to main

### Root Cause 2: Missing CARGO_REGISTRY_TOKEN Secret

**Symptom:** `publish-to-crates.mjs` fails with:
```
Error: CARGO_REGISTRY_TOKEN environment variable is not set
Crate link-assistant-agent does not exist on crates.io yet (first publish)
```

**CI Run:** [24328743522](https://github.com/link-assistant/agent/actions/runs/24328743522) (2026-04-13 06:19 UTC)

**Root Cause:** The `CARGO_REGISTRY_TOKEN` secret was not configured in the repository settings. The workflow used `${{ secrets.CARGO_REGISTRY_TOKEN || secrets.CARGO_TOKEN }}` which resolves to empty string when neither secret exists.

**Fix (previously applied in commit f675128):** Added `CARGO_TOKEN` as a fallback in both the workflow YAML and the publish script. The underlying issue (no secret configured) requires manual action in GitHub repository settings.

## Requirements from Issue

| # | Requirement | Status |
|---|-------------|--------|
| 1 | Double check latest changes before publishing | Fixed — fetch/rebase added |
| 2 | Handle changesets properly, merge into single release | Already working via changelog fragment system |
| 3 | Update changelog | Already working via `rust-collect-changelog.mjs` |
| 4 | Update Cargo.toml | Already working via `rust-version-and-commit.mjs` |
| 5 | Compare with template best practices | Analyzed — see below |
| 6 | Download logs and create case study | This document |
| 7 | Report issues to related repos | See below |

## Comparison with Template Repository

Reference: [rust-ai-driven-development-pipeline-template](https://github.com/link-foundation/rust-ai-driven-development-pipeline-template)

### Features in Template Not Yet in This Repo

| Feature | Template | This Repo | Impact |
|---------|----------|-----------|--------|
| `detect-changes` job | Yes | No | Template only runs CI when relevant files change; this repo uses path filters instead (sufficient for monorepo) |
| `version-check` job | Yes | No | Template prevents accidental manual version bumps in PRs |
| Code coverage (cargo-llvm-cov) | Yes | No | Low priority — can be added later |
| Deploy docs to GitHub Pages | Yes | No | Low priority — useful when API docs exist |
| Pre-commit hooks (.pre-commit-config.yaml) | Yes | Uses .husky | Equivalent functionality |
| Push race condition handling | **No** | **Now Fixed** | Template has the same bug — reported as issue |

### Key Observation

The template's `version-and-commit.rs` also does a plain `git push` without pull/rebase (line 497), meaning it would fail the same way in a multi-workflow repository. This is acceptable for the template because it's single-workflow (only one release job can run), but it's a latent bug if the template is extended. An issue has been filed.

## Files Changed

- `scripts/rust-version-and-commit.mjs` — Added fetch/rebase before commit, push retry with pull --rebase
- `.github/workflows/rust.yml` — Added `concurrency: { group: release-main }` to auto-release and manual-release jobs
- `.github/workflows/js.yml` — Added `concurrency: { group: release-main }` to release and instant-release jobs

## CI Logs

- [rust-24331934514.log](./rust-24331934514.log) — Non-fast-forward push failure
- [rust-24328743522.log](./rust-24328743522.log) — CARGO_REGISTRY_TOKEN not set failure
5 changes: 5 additions & 0 deletions js/.changeset/fix-release-concurrency.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@link-assistant/agent': patch
---

Added shared concurrency group to JS release jobs to prevent race condition with Rust CI/CD pushes to main
9 changes: 9 additions & 0 deletions rust/changelog.d/20260413_fix_push_race_condition.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
bump: patch
---

### Fixed

- Fixed git push race condition in Rust CI/CD auto-release that caused non-fast-forward rejection when JS CI pushed concurrently
- Added fetch/rebase before commit and push retry with pull --rebase (up to 3 attempts)
- Added shared concurrency group (`release-main`) across Rust and JS release jobs to serialize pushes to main
40 changes: 36 additions & 4 deletions scripts/rust-version-and-commit.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,8 @@ ${newEntry}
}
}

const MAX_PUSH_RETRIES = 3;

async function main() {
try {
// Configure git
Expand All @@ -254,6 +256,21 @@ async function main() {
'git config user.email "github-actions[bot]@users.noreply.github.com"'
);

// Pull latest changes from remote before starting
console.log('Fetching latest changes from origin/main...');
exec('git fetch origin main');

const localHead = exec('git rev-parse HEAD');
const remoteHead = exec('git rev-parse origin/main');

if (localHead !== remoteHead) {
console.log(
`Remote main has advanced (local: ${localHead.slice(0, 7)}, remote: ${remoteHead.slice(0, 7)})`
);
console.log('Rebasing on remote main to incorporate changes...');
exec('git rebase origin/main');
}

const current = getCurrentVersion();
const newVersion = calculateNewVersion(current, bumpType);

Expand Down Expand Up @@ -313,10 +330,25 @@ async function main() {
exec(`git tag -a rust-v${newVersion} -m "${tagMsg.replace(/"/g, '\\"')}"`);
console.log(`Created tag rust-v${newVersion}`);

// Push changes and tag
exec('git push');
exec('git push --tags');
console.log('Pushed changes and tags');
// Push with retry: if another CI job (e.g. JS release) pushed to main
// concurrently, pull --rebase and retry
for (let attempt = 1; attempt <= MAX_PUSH_RETRIES; attempt++) {
try {
exec('git push');
exec('git push --tags');
console.log('Pushed changes and tags');
break;
} catch (pushError) {
if (attempt < MAX_PUSH_RETRIES) {
console.log(
`Push failed (attempt ${attempt}/${MAX_PUSH_RETRIES}), pulling and retrying...`
);
exec('git pull --rebase origin main');
} else {
throw pushError;
}
}
}

setOutput('version_committed', 'true');
setOutput('new_version', newVersion);
Expand Down
Loading