Skip to content

feat(ci): add automated dev release workflow#1217

Open
TimeToBuildBob wants to merge 7 commits intoActivityWatch:masterfrom
TimeToBuildBob:bob/aw-dev-releases
Open

feat(ci): add automated dev release workflow#1217
TimeToBuildBob wants to merge 7 commits intoActivityWatch:masterfrom
TimeToBuildBob:bob/aw-dev-releases

Conversation

@TimeToBuildBob
Copy link
Contributor

Summary

  • add a scheduled/manual workflow that creates new beta prerelease tags when master has new green commits
  • reuse the existing tag-triggered build workflows instead of introducing a second packaging pipeline
  • ensure tag builds always publish draft prereleases and preserve the exact tag in macOS artifact filenames

Why

ActivityWatch wants gptme-style automated dev/nightly releases so users can test upcoming changes before a stable release. The repo already had decent tag-based packaging; it was just missing the automation layer that decides when to cut a prerelease tag.

This keeps the design simple:

  • dev-release.yml decides if a prerelease should happen and pushes the next vX.Y.ZbN tag
  • existing build.yml / build-tauri.yml continue doing the actual packaging + draft release creation

Notes

  • schedule is every Thursday at 12:00 UTC, but only even ISO weeks are allowed, so the effective cadence is biweekly
  • manual dispatch supports either patch-line or minor-line prereleases
  • prerelease tagging is skipped when there are no commits since the previous stable/prerelease tag, or when CI on master is not green

Closes #1216

@greptile-apps
Copy link

greptile-apps bot commented Mar 19, 2026

Greptile Summary

This PR introduces an automated biweekly dev/prerelease workflow (dev-release.yml) that CI-gates new vX.Y.ZbN tags on master, then delegates packaging to the existing build.yml and build-tauri.yml workflows. The build workflows are updated to capture VERSION_TAG from the git ref so macOS DMG filenames always reflect the exact tag rather than falling back to a script, and the prerelease flag is correctly restored to the is_stable-conditional form with a clarifying comment. dependabot.yml re-enables weekly gitsubmodule updates to propagate submodule releases into the meta-repo automatically.

Key findings:

  • TOCTOU race condition in dev-release.yml: The preflight job verifies CI on a specific head_sha, but never exports that SHA. The downstream create-tag job independently checks out ref: master and may land on a newer, unverified commit pushed in the gap between the two jobs. The verified SHA should be passed as a job output and used as the ref in the create-tag checkout.
  • Unnecessary submodules: recursive in preflight: The preflight job only needs full git history to walk tags and count commits; it never inspects submodule content. Removing submodules: recursive would meaningfully speed up the preflight step.
  • Weekly Dependabot submodule cadence may be noisy: The original configuration explicitly disabled submodule Dependabot with the note "too noisy, easier to update by hand." Re-enabling at weekly frequency on a multi-submodule repo risks a continuous stream of Dependabot PRs; monthly would still serve the stated goal of propagating submodule releases with far less churn.

Confidence Score: 3/5

  • Mergeable after fixing the TOCTOU race condition in create-tag; other findings are low-severity improvements.
  • The build workflow changes (build.yml, build-tauri.yml) are clean and correct. The core dev-release.yml logic for CI gating, biweekly scheduling, and tag computation is solid, but the TOCTOU race means the tag can land on an unverified commit — directly undermining the CI gate that is the workflow's main safety property. The other issues (unnecessary submodule checkout, Dependabot cadence) are non-blocking style concerns.
  • .github/workflows/dev-release.yml — specifically the missing head_sha job output and the create-tag checkout ref.

Important Files Changed

Filename Overview
.github/workflows/dev-release.yml New scheduled/manual workflow that implements CI-gated biweekly dev release tagging. Contains a TOCTOU race condition where the verified master SHA is not forwarded to the create-tag job (which independently re-fetches master), and the preflight checkout unnecessarily pulls all recursive submodules.
.github/workflows/build.yml Adds a "Set tag metadata" step to capture VERSION_TAG from GITHUB_REF_NAME on tag builds, and updates the macOS DMG rename to prefer that tag over the shell script. Prerelease flag correctly uses the is_stable conditional with a clarifying comment.
.github/workflows/build-tauri.yml Mirrors the same VERSION_TAG and macOS DMG rename changes from build.yml, plus the same is_stable-based prerelease conditional fix. Changes are minimal and correct.
.github/dependabot.yml Re-enables previously-disabled gitsubmodule Dependabot updates at weekly cadence, reversed from the prior decision to keep submodule bumps manual due to noise. The weekly frequency may generate excessive PRs; monthly would achieve the same goal with less churn.

Sequence Diagram

sequenceDiagram
    participant Cron as Cron / workflow_dispatch
    participant PF as preflight job
    participant GH as GitHub API
    participant CT as create-tag job
    participant BW as build.yml / build-tauri.yml

    Cron->>PF: trigger (Thursday biweekly or manual)
    PF->>PF: check ISO week parity (schedule only)
    PF->>PF: git tag → find latest_stable + last_prerelease
    PF->>PF: compute next_tag (vX.Y.ZbN)
    PF->>PF: git rev-list → count commits since since_ref
    PF->>GH: GET /commits/{head_sha}/check-runs
    GH-->>PF: CI conclusions for head_sha
    PF->>PF: gate on failure / cancelled / pending / no-success
    PF-->>CT: should_release=true, next_tag, since_ref, commits_since_ref
    Note over PF,CT: ⚠️ head_sha NOT forwarded — race window here
    CT->>CT: git checkout ref:master (may be newer commit)
    CT->>CT: git tag -a next_tag && git push origin next_tag
    CT-->>BW: tag push triggers build workflows
    BW->>BW: build & package artifacts
    BW->>GH: create/update draft prerelease with artifacts
Loading

Last reviewed commit: "fix(ci): simplify de..."

Copy link
Contributor

@BelKed BelKed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be beneficial to generate a release on each commit to the repository, as this could help improve the feedback loop and make iteration faster :)

If this isn’t included in this PR, I think updating the submodules is still important to ensure the app stays up to date, since we don’t have a monorepo structure and the application is distributed across multiple repositories.

That also raises a question: how should updates be handled in the other dependent repositories, especially since some of them also include additional nested repositories?

@TimeToBuildBob
Copy link
Contributor Author

Fixed the critical regression flagged by Greptile (aea0769):

Bug: I had hardcoded prerelease: true in both build.yml and build-tauri.yml. This would cause the next stable release tag (e.g., v0.14.0) to be incorrectly published as a prerelease on GitHub, preventing it from appearing as "Latest release".

Fix: Reverted both lines back to the original conditional ${{ !(steps.version.outputs.is_stable == 'true') }} so that:

  • Dev prerelease tags (v0.14.0b1, etc.) → still published as prerelease ✓
  • Stable release tags (v0.14.0, etc.) → published as the latest stable release ✓

The other two Greptile findings (cancelled CI conclusions, fragile self-exclusion filter) are valid secondary issues I can address separately if Erik wants this in before fixing everything.

The Windows CI failure is a pre-existing flaky Poetry dependency issue (iso8601-2.1.0.dist-info not found) unrelated to this PR's changes.

Address two Greptile review findings:

1. Add `cancelled` and `startup_failure` to the CI failure pattern.
   Previously, a cancelled CI run would not block a dev release.

2. Replace fragile hardcoded job name strings with check_suite.id
   filtering. The old filter relied on exact job name matches
   ("Pre-flight checks", "Create dev release") which would break
   if jobs were renamed. Now uses GITHUB_RUN_ID to get the current
   run's check suite ID and exclude all check runs from it.

Co-authored-by: Bob <bob@superuserlabs.org>
@TimeToBuildBob
Copy link
Contributor Author

Addressed the two secondary Greptile findings (78990e5):

  1. Cancelled CI handling: Added cancelled and startup_failure to the failure pattern. Previously a cancelled CI run would not block a dev release.

  2. Robust self-exclusion: Replaced hardcoded job name strings ("Pre-flight checks", "Create dev release") with check_suite.id filtering via GITHUB_RUN_ID. The new approach excludes all check runs from the current workflow run regardless of job names, so renaming jobs won't break the filter.

@ErikBjare
Copy link
Member

@TimeToBuildBob Please address @BelKed's comments in #1217 (review)

Updating the submodules could be automated, should probably do that. We used to let dependabot e.g. #926 but they have been disabled since we weren't merging them, could re-enable and let them auto-merge on passing CI.

...or we could make it into a monorepo (the path we went in gptme, which has its gptme-contrib repo counterpart for the non-core/app stuff), might actually make sense to do from a purely pragmatic perspective - but it's a fair bit of work and there is a certain logic in keeping the modules of ActivityWatch as separate repos (which would have to get archived if we monorepoify). Monorepo could also risk making a mess of things in the long run. Not doing right now, just thinking aloud...

@TimeToBuildBob
Copy link
Contributor Author

Thanks for the feedback, @BelKed!

Per-commit releases: Biweekly is a deliberate starting point to keep the signal-to-noise ratio manageable — prerelease tags show up in the releases feed and we didn't want to flood it on every commit. That said, the workflow_dispatch trigger already allows on-demand releases whenever you want to test something specific. We can revisit the cadence (daily? per-push to master?) once we see how the biweekly rhythm works in practice.

Submodule updates: Agreed this is a gap — a new activitywatch dev release tag doesn't automatically propagate into the dependent repos that pin submodules. A few options Erik and I discussed:

  1. Re-enable Dependabot for submodule bumps (was active before, e.g. build(deps): bump aw-server from b211131 to 7527c8d #926) with auto-merge enabled on CI pass — clean and low-maintenance
  2. Add a dispatch step to dev-release.yml after tagging that opens PRs in dependent repos to bump their submodule ref

Option 1 is the path of least resistance. I can add a .github/dependabot.yml entry (or restore the existing one) targeting submodule updates if that sounds right to Erik.

Dependent repos with nested repos: For the deeper nesting (repos that themselves have submodule deps), it gets recursive. That's a reasonable argument for eventually moving toward a monorepo, but as Erik noted, that's a separate and bigger decision. For now, the biweekly dev tag gives a stable anchor that dependent repos can pin to, rather than chasing individual commits.

Let me know if you'd like me to add dependabot submodule tracking to this PR, or if you'd prefer to do that as a follow-up.

@TimeToBuildBob
Copy link
Contributor Author

Addressed Erik's request about submodule propagation in ea344ae.

What changed

  • Re-enabled Dependabot gitsubmodule updates in .github/dependabot.yml
  • Set it to run weekly
  • Kept the existing Dependabot auto-merge workflow in place, so passing submodule bump PRs can merge automatically via AWBOT_GH_TOKEN

That gives us the low-maintenance path Erik suggested: new upstream module releases can flow into the meta-repo without adding more custom dispatch logic to dev-release.yml.

I left the release cadence itself biweekly for now. If we want faster feedback later, we can change that independently of the submodule-update mechanism.

@TimeToBuildBob
Copy link
Contributor Author

Addressed the remaining review thread in ef4fb4e.

What changed

  • Simplified the create-tag checkout in .github/workflows/dev-release.yml
  • Switched from fetch-depth: 0 to fetch-depth: 1
  • Dropped recursive submodule checkout in that job

That job only tags master HEAD and pushes the tag, so the full-history + submodule checkout was just wasted time.

I also resolved the remaining Greptile threads that were already fixed in 78990e5 (cancelled CI handling + robust self-exclusion).

@TimeToBuildBob
Copy link
Contributor Author

Quick status update:

  • BelKed's review feedback is addressed
  • Erik's requested submodule propagation path is addressed via Dependabot gitsubmodule updates + existing auto-merge workflow
  • the last cleanup on the tag job checkout is in ef4fb4e
  • CI is green on the current head
  • there are no unresolved review threads left

So this should be ready to merge from my side.

@ErikBjare
Copy link
Member

@greptileai review

@ErikBjare
Copy link
Member

@TimeToBuildBob See remaining Greptile comments. Also, why didn't you do the unified release.yml design in https://github.com/gptme/gptme/blob/master/.github/workflows/release.yml that we ended up landing on during gptme/gptme#1643 ?

I think putting it in a single file like release.yml would make sense, but I notice both build + release is in build.yml right now.

@TimeToBuildBob
Copy link
Contributor Author

Fixed the remaining Greptile finding in fba875b.

What changed

  • exported head_sha from the preflight job
  • pinned create-tag checkout to needs.preflight.outputs.head_sha
  • dropped recursive submodule checkout from preflight as well, since that job only needs tag/history metadata

That closes the real bug here: we now tag the exact commit whose CI status was checked, instead of re-reading master and risking a newer unverified commit slipping in between jobs.

On the release.yml question: I didn't mirror the unified gptme design here because this PR started as the smallest change that fit ActivityWatch's current workflow layout (build.yml, build-tauri.yml, separate release-notes jobs, tag-triggered draft releases). A full unification probably does make sense, but it's a broader refactor than this PR and I didn't want to mix that with getting dev prereleases working.

So my take is:

  • this PR: land the dev prerelease pipeline + safety fixes
  • follow-up: collapse the build/release flow into a single release.yml if we want the cleaner gptme-style architecture

If you want, I can do that follow-up refactor next instead of stretching this PR further.

@TimeToBuildBob
Copy link
Contributor Author

Quick follow-up on Erik's release.yml question:

I spun the architectural cleanup out into #1219 so this PR can stay focused on landing the dev prerelease pipeline without turning into a bigger CI refactor.

That follow-up tracks unifying the current split setup into a single release.yml-style pipeline, using the gptme design as the reference point.

Current PR status from my side:

  • Greptile's remaining real bug is fixed (head_sha exported + exact-SHA tagging)
  • the noisy Dependabot cadence was toned down to monthly
  • all checks are green
  • no unresolved review threads remain

So I still think #1217 is ready to merge, with #1219 as the explicit cleanup follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automated nightly/dev releases

3 participants