Introduce cargo-ox-check: unified Rust build/CI scaffolding#33
Draft
martin-kolinek wants to merge 77 commits into
Draft
Introduce cargo-ox-check: unified Rust build/CI scaffolding#33martin-kolinek wants to merge 77 commits into
martin-kolinek wants to merge 77 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wires clap (derive) with the single 'update' subcommand and flags --backend (repeatable), --no-backends (mutually exclusive with --backend), and --dry-run. Adds anyhow, thiserror, tracing, tracing-subscriber deps. main.rs strips the 'ox-check' token cargo injects for subcommand binaries. run_update is a no-op that logs its inputs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the backend module with Backend enum (GitHub, Ado), URL parsing (https/scp/ssh forms), git-config invocation for origin, and a resolve() function that implements the CLI resolution order: --no-backends > --backend > autodetect. Unit tests cover URL parsing edge cases, name parsing, and resolution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the workspace module: find_workspace_root() walks up from a start path to the nearest [workspace] ancestor (falling back to the nearest [package] for single-crate repos), and load_workspace() parses the manifest with toml_edit and resolves members. Supports literal member entries and 'crates/*'-style trailing globs (the only form observed in surveyed repos). Adds toml_edit to workspace deps. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the manifest module implementing the sidecar file documented in updates.md. Manifest::load() returns Manifest::default() for missing files. Manifest::save() writes atomically via temp+rename. to_toml() serializes deterministically with sorted entries (BTreeMap-backed), always ending in a trailing newline so diffs are minimal. Rejects schema versions newer than 1; missing/duplicate entries are hard errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds checksum and decision modules. checksum_bytes/checksum_str return the canonical 'sha256:<hex>' string used throughout the codebase. decide() implements the table from updates.md section 5 with five outcomes: InSync, Skipped (opted out), Write, Propose, LeaveAlone. Includes should_emit_proposed_for_opt_out() to handle the empty-stub case where we still proactively surface upstream churn. Adds sha2 to workspace deps. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the region module: find_region(), upsert_region(), and render_region() implement the sentinel-delimited managed-region machinery for any host file. Supports two comment syntaxes (# for Justfile/TOML/YAML and // for future hosts). Recognizes indented sentinels (needed for YAML). Detects malformed regions (duplicate opener, missing closer, closer-before-opener) and reports clean errors. Treats whitespace-only bodies as empty (= opt-out signal). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the plan module: PlanItem encapsulates one accumulated decision plus the rendered content (or spliced host body for regions). Plan::apply() writes owned files, splices region updates, writes .ox-check-proposed siblings, and returns an updated manifest. Plan::summary() renders a stable line-oriented digest; Plan::dry_run_exit_code() returns 1 iff any item would write. For region proposals the proposed file lives next to the *host* (host.ox-check-proposed), and per updates.md section 7 it contains the full spliced host file rather than just the region body. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the emit module (mod, owned_file, local). plan_owned_file() drives one file through the decision algorithm: reads the disk content, detects opt-out (empty/whitespace-only), and produces a PlanItem. plan_tools_just() is the first concrete emitter, embedding templates/justfiles/ox-check/tools.just via include_str! and dispatching through the owned-file driver. The template is wrapped in a sentinel-delimited block per the design convention but is itself an owned file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the full catalog from checks.md as one recipe per check (ox-check-<name>): all pr-fast, pr-test, pr-mutants, nightly-runtime, and nightly-exhaustive members. pr-title is the lone [script('pwsh')] block; everything else is a one-line cargo invocation gated by an ox-check-tools-check or _ox-check-require dependency. plan_checks_just() drives it through the same owned-file path as tools.just.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the two remaining files of the justfiles/ox-check/ tree. groups.just defines the seven check-groups (3 pr + 4 nightly) as just-recipe dependency lists pointing at the per-check recipes from commit 9. tiers.just aggregates them into ox-check-pr, ox-check-nightly, and ox-check-full. plan_local_just_tree() bundles all four file emitters into a single helper for the run driver. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the managed_region driver — the region equivalent of plan_owned_file — and the first managed-region emitter for the user's Justfile. plan_justfile_imports() inserts the four 'import' lines plus the 'alias ox-check := ox-check-pr' line into the ox-check-imports region. Creates the Justfile if absent; appends the region if the file has user content; replaces just the region body otherwise. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the cargo_toml emitter producing managed regions for the lint catalog. For multi-crate workspaces: one ox-check-workspace-lints region in the root Cargo.toml ([workspace.lints] in dotted-key form) plus an ox-check-lints region with 'workspace = true' in each member. For single-crate repos: one ox-check-lints region with the full catalog at [lints] scope. The dotted-key form lets users extend the same scope outside the sentinels (TOML forbids re-declaring [workspace.lints.clippy]). Includes a round-trip test that splices the body into a real Cargo.toml and re-parses with toml_edit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds shared_configs emitter producing one managed region per file. Bodies stay intentionally small so most adopters never need to override: deny.toml carries a permissive SPDX allowlist plus advisory/yanked rules, rustfmt.toml sets edition/width/newline-style only, and .delta.toml configures the root files that invalidate impact-scoping. Each can be opted out by emptying its region — no special flag needed. Round-trip test parses every spliced body through toml_edit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wires the run module to actually drive the update algorithm: discovers the workspace root, loads the manifest, resolves backends, builds the full plan from all local emitters (just tree + Justfile imports + Cargo.toml lints + shared configs), applies (or dry-runs) it, and rewrites .ox-check.lock. Exposes run_update() taking an explicit start directory so integration tests can drive the algorithm without std::process::exit. Adds e2e tests covering first-run write, idempotent second run, dry-run abstinence, opt-out via empty region, and user-edit-with-template-unchanged leave-alone. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the first layer of the GitHub Actions backend. Two shared composite actions live as static template files: ox-check-setup installs just and the catalog tools; ox-check-impact runs cargo-delta and emits excludes/skip outputs for the impact job. The seven per-group composite actions are rendered programmatically (render_group_action) since they differ only by group name; each takes excludes/skip inputs from the impact job and invokes just ox-check-<group>. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the wiring layer: ox-check-pr-impl.yml and ox-check-nightly-impl.yml. Both are workflow_call entry points. PR runs an impact job (cargo-delta) then fans out into pr-fast, pr-test (matrix across test_os input), and pr-mutants jobs that each consume the excludes/skip outputs. Nightly skips the impact job (slow checks always run on main) and fans out into the four nightly groups, uploading the coverage lcov artifact from the Linux test leg. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the two root workflows (ox-check-pr.yml, ox-check-nightly.yml) that own triggers, permissions, and concurrency, then call the reusable workflows. plan_github_backend() bundles all 13 files (2 shared actions + 7 group actions + 2 reusable workflows + 2 root workflows). The run driver dispatches to it when Backend::GitHub is in the resolved set. After this commit --backend github is fully functional. Integration test creates a workspace and asserts the full .github tree is present after one update; a second test confirms idempotency. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the first layer of the ADO backend. setup.yml installs just + catalog tools; impact.yml runs cargo-delta and publishes excludes/skip variables via ##vso[task.setvariable] with isOutput=true so the stages template can consume them as stage outputs. The seven per-group step templates are rendered programmatically (render_group_step); each takes excludes/skip parameters, includes setup.yml, and invokes the matching just recipe with OX_CHECK_EXCLUDES set. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds pr.yml and nightly.yml stages templates. PR stages: impact stage publishes excludes/skip outputs that downstream stages (pr_fast, pr_test, pr_mutants) consume via stageDependencies. pr_test fans out across linux/windows jobs. Every group stage condition is succeededOrFailed() so a failing pr-fast doesn't gate pr-test. Nightly stages: four parallel stages with empty dependsOn arrays, plus an lcov artifact publish from the linux nightly-test leg. Both templates expose linuxPool/windowsPool object parameters so users can override pools when wrapping in a 1ESPT pipeline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds ox-check-pr.yml and ox-check-nightly.yml root pipelines. PR pipeline is PR-only (trigger: none, pr triggers on main); nightly carries a daily cron schedule. Both pass default vmImage-based pools to the stages templates; users wrapping in 1ESPT override these by replacing the root pipeline. plan_ado_backend() bundles all 13 files. The run driver dispatches to it when Backend::Ado is in the resolved set. After this commit --backend ado is fully functional. Integration tests cover ado-only and ado+github combined runs for idempotency. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds tests/schemas.rs running external validators against the full emitted tree from a fresh workspace. Validators (taplo for TOML, actionlint for GH workflows, just --list for the Justfile) are invoked via Command; any validator not installed produces a 'skipping' message rather than a test failure, so the suite works on every developer machine while enforcing schema correctness in CI. ADO YAML has no public standalone validator we depend on, so the ado test confirms only structural properties (no tabs, even-space indentation). Also adds 'set unstable' to checks.just so just accepts the [script('pwsh')] attribute on pr-title.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds .github/workflows/regenerate-check.yml: builds cargo-ox-check from the PR branch and runs 'cargo ox-check update --dry-run' against the repo, failing the PR if the in-tree state diverges from what the templates would render. This is the primary dogfooding mechanism from verification.md section 3. Note: the workflow will report a divergence until a maintainer runs the actual bootstrap step (regenerate the workspace lints region in dotted-key form, write the justfiles/ox-check tree, etc.) — that is a separate human-driven activity, intentionally not folded into this implementation PR. The verification.md bootstrap section covers the procedure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expands lib.rs doc-comments into a publishable-quality crate description covering the what/install/usage/daily-driver/customization story, all linking back to the docs/design/ files. Adds a placeholder README.md following the cargo_heather pattern (auto-generated from doc-comments by 'just readme' from the repo root). 'cargo publish --dry-run -p cargo-ox-check --allow-dirty' completes successfully: 49 files, 341 KiB packaged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Swaps the error type to match the convention used by the other crates in this repo (cargo_heather, cargo_ensure_no_cyclic_deps). Function signatures now return Result<T, ohno::AppError>; error construction uses ohno::app_err! and ohno::bail!; context attachment uses IntoAppError::into_app_err / into_app_err_with. The allowed_external_types whitelist is updated accordingly. All 155 tests still pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Empty content has a stable checksum that no template ever produces, so emptying a file or region body always lands in the standard D != L branches. The steady-state opt-out case (after at least one prior render) reaches LeaveAlone naturally — silent — and a template change reaches Propose — surfaces upstream churn. Both outcomes preserve the user's empty stub, matching the design contract. The only behavior change is the edge case where the user creates an empty file *before* the first ox-check update (no manifest entry yet): instead of a silent skip, ox-check now writes a one-time .ox-check-proposed sibling showing what the template would render. That's arguably better UX since it documents what the user is opting out of. Removes: - DecisionInputs::emptied field - Decision::Skipped variant - should_emit_proposed_for_opt_out helper - the trim-empty detection in owned_file and managed_region drivers All 155 tests still pass. The behavior the tests covered survives: opt-out tests now assert LeaveAlone (when template unchanged) or Propose (when changed) instead of Skipped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tools.just, checks.just, groups.just, tiers.just are owned files — the manifest tracks them by full-file checksum. Sentinels are reserved for managed regions inside user-composed hosts (Justfile, Cargo.toml, deny.toml, rustfmt.toml, .delta.toml) where ox-check carves out a section within other content. The 'Owned by cargo-ox-check' advisory comment stays as a one-line notice for human readers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…files Three coupled cleanups: 1. Add justfiles/ox-check/mod.just as the single entry point. It imports the four sibling .just files and defines 'alias ox-check := ox-check-pr'. The user's Justfile region is now a single 'import justfiles/ox-check/mod.just' line — everything else is owned files the user never edits directly. 2. Move the alias out of the user's Justfile region. The user no longer has any recipe content in their Justfile; mod.just owns it. Easier to evolve (renaming or retargeting the alias is a template update, not a managed-region change). 3. Move every region body out of Rust constants into templates/regions/*.toml and templates/regions/*.just. cargo-lints-body.toml carries the dotted-key catalog with no host-specific header; cargo_toml.rs prepends [workspace.lints] or [lints] based on the manifest shape. Eliminates ~50 lines of inline string and array-of-tuples plumbing. All 152 unit + 4 integration tests still pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The two remaining YAML-in-Rust blobs (render_group_action, render_group_step) used format! to splice the group name into a multi-line string. Replaced with templates/github/group-action.yml and templates/ado/steps/group.yml carrying a __GROUP__ placeholder; Rust now just include_str!s the file and runs a single .replace(). __GROUP__ chosen over a curly-brace placeholder so it can't collide with GitHub Actions' expression syntax inside the same file. All templates now live as files on disk; no YAML or TOML appears inline in .rs sources. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drops the hand-written placeholder in favor of the standard ox-tools README generated by 'just readme' (uses ../README.j2 with the crate-level doc-comments from src/lib.rs). Adds the standard crates.io/docs.rs/MSRV/CI/Coverage/License badge row and the project footer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two small per-audit fixes bundled: - checks.just ox-check-llvm-cov: add a third 'cargo llvm-cov report --html' step after the cobertura emit. Lands at target/coverage/html/index.html; no CI consumer, purely a local affordance for inspecting coverage on a developer machine. - cargo-lints-body.toml: add clippy.expect_used = 'warn' and rustdoc.broken_intra_doc_links = 'warn' to the catalog. Both were documented in design.md / updates.md but absent from the actual emitted body. expect_used paired with the existing unwrap_used closes a common 'we don't trust .expect either' gap. broken_intra_doc_links catches intra-doc links to nonexistent items at clippy time. Snapshots regenerated for all three backend combinations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per audit C5. The ox-check-mutants recipe was hard-coded to origin/main when BASE_REF wasn't set; repos using 'master' as their canonical branch saw the recipe fail with 'unknown revision' on local invocations. Resolution order: BASE_REF env > origin/main > origin/master > error with a clear hint. Matches the ADO nightly schedule's [main, master] branch list and the resolution policy described in checks.md section pr-mutants. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per audit A11 / B7. Several recipes (fmt, clippy, doc-build, doc-test, examples, miri, bench) had inconsistent or missing tool-version dependencies. Some used ox-check-tools-check (which now loops the whole catalog — gross overhead per recipe); others declared no require at all. Each recipe now declares exactly what it needs: - fmt / clippy: no require. cargo-fmt and cargo-clippy are rustup components bundled with the toolchain; if cargo is on PATH, they're on PATH too. - doc-build / doc-test / examples / miri / bench: no require. All use toolchain-bundled cargo subcommands (rustdoc, test, build, miri component, bench). The catalog tools they depend on are unrelated. - pr-title: gains (_ox-check-require pwsh) to surface the missing-pwsh case with the right install hint instead of a cryptic 'pwsh: command not found' error. ox-check-tools-check stays as the explicit 'verify everything in the catalog' smoke test, which is what its name implies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per audit A7 and github.md section 8. The setup composite now caches cargo home + target/ with a key composed of OS, rustc version, lockfile-family hash, and the tool-minimums.txt hash. Cache invalidates cleanly when: - the runner OS changes (independent cache namespaces); - the rust toolchain bumps (rustc --version takes that drift); - any of Cargo.lock / .cargo/config.toml / rust-toolchain.toml change (dependency churn or toolchain pin); - tool-minimums.txt changes (Phase 2 catalog bumps invalidate the installed-tool cache as intended). Cache paths: ~/.cargo/registry/cache, ~/.cargo/registry/index, ~/.cargo/bin (for the catalog-installed tools), and target/ (per-recipe cache scope handled implicitly by cargo's incremental compilation). restore-keys fall back through partial matches so a rustc bump still gets partial reuse. Snapshots regenerated for the github backend. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per audit A7 and ado.md section 7. Setup step now uses Cache@2 with a key shape that mirrors the GitHub actions/cache side from C15: ox-check-v1 | <agent OS> | rust<version> | Cargo.lock | .cargo/config.toml | rust-toolchain.toml | tool-minimums.txt ADO's Cache@2 key syntax differs from GH (pipe-separated fragments, lockfile-detection inline), but the semantic content is identical. restoreKeys provide the same prefix-fallback ladder as restore-keys on GH. Cache paths: cargo registry/index/bin + target/, all under HOME. On ADO-hosted Linux/macOS agents HOME is /Users/runner or /home/vsts; on Windows it's C:\Users\VssAdministrator. ADO's Cache@2 normalizes the path resolution. Snapshots regenerated for the ado backend. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per audit A8 and 0001.md Phase 6 C17. Adds four fixture directory trees plus a tests/update.rs runner that stages each fixture into a tempdir and runs run_update + scenario-specific assertions. Fixtures: - single-crate/: bare [package] (no [workspace]); asserts the per-crate lints region is emitted (not the workspace one), the full justfiles/ox-check/ tree is written, and a second run is a no-op. - opt-outs/: workspace where the user empties the rustfmt managed region after the first run; asserts the rustfmt region item lands as LeaveAlone and the file stays empty. - customized/: workspace where the user edits inside the rustfmt managed region; asserts LeaveAlone with the user content preserved verbatim. - migration/: workspace with a pre-existing Justfile, deny.toml, and [profile.release] in Cargo.toml; asserts ox-check splices its regions and the user content survives the migration. The tests are complementary to the in-memory scenarios in src/run.rs: the fixtures are real directory trees on disk that reviewers can read and copy when designing migration paths or onboarding scenarios. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per audit D5 and 0001.md Phase 6 C18. Brings the verification doc in line with the actual test layers landed in C17: - Section 2.3 (fixtures): replace the aspirational determinism/manifest-consistency promises with what tests/update.rs actually asserts (single-crate emission, opt-out LeaveAlone, customized LeaveAlone preservation, migration splicing) plus the idempotence checks. - Section 2.3 (header): fix the duplicate '### 2.3' on schema validation (was 2.3, should be 2.4); renumber the manual-release section to 2.5 with downstream references in 5. updated. - Table in section 6: tighten the tests/update.rs description from the stale 'determinism/consistency' summary to the per-scenario reality. No code change; doc-only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…sify checks
Followup to P3. cargo-delta defines three concentric impact tiers (modified ⊂ affected ⊂ required) where 'required' = 'affected ∪ workspace-internal transitive deps' — the right scope for tools that resolve through the dep graph. The original P3 work conflated 'required' with 'always run on the whole workspace' and dropped the OX_CHECK_INCLUDE_REQUIRED env var entirely, mis-classifying doc-build / cargo-hack / udeps and obscuring the distinction between 'no workspace scope' (deny, audit, pr-title) and 'workspace-wide because the tool needs the full dep graph' (cargo doc, cargo-hack).
Bucket assignments going forward:
- modified: fmt, license-headers, cargo-sort, ensure-no-cyclic-deps, ensure-no-default-features, readme-check, spellcheck
- affected: clippy (deliberate deviation from cargo-delta's recommendation — see comment in checks.just; a change in a crate can cause clippy issues in dependents), llvm-cov, doc-test, examples, mutants, miri, careful, semver-check, external-types, bench
- required: doc-build, udeps, cargo-hack (feature powerset)
- unscoped (no env var): pr-title, deny, audit, aprz, mutants-full
Changes:
- templates/{github,ado}/impact-action.yml + impact.yml: emit a third output include_required, formatted with the same '--package …' or '--skip' sentinel as the other two.
- templates/github/group-action.yml, templates/ado/steps/group.yml: take include_required input and export OX_CHECK_INCLUDE_REQUIRED.
- templates/github/pr-impl-workflow.yml, templates/ado/pr-stages.yml: thread include_required through every group invocation.
- templates/justfiles/ox-check/checks.just: reclassify recipes per the table above; new skip guards on doc-build (required), udeps (required), cargo-hack (required); rewrite the header comment to describe the four-bucket model; semver-check / external-types switched from unscoped-required to affected.
- docs/design/checks.md section 5: rewrite the env-var table with the four-bucket model and explicit bucket-per-check listing; document the clippy-on-affected deviation; clarify what --skip is.
- docs/design/github.md sections 4-6: replace exclude-based YAML with include-based YAML; drop the impact_skip plumbing (subsumed by the per-var --skip sentinel); add include_required everywhere; rewrite the impact-action output table.
- docs/design/ado.md sections 4-5: parallel rewrite for the ADO templates; renamed parameters from excludeNot* / impactSkip to includeModified / includeAffected / includeRequired; renamed delta.exclude_not_* outputs to compute.include_*.
- docs/design/local.md section 4: rewrite as a three-env-var, four-bucket table; explain the --skip sentinel; replace the old 'OX_CHECK_IMPACT_SKIP early-return hint' subsection with one on the --skip sentinel; update the local-scoped-run example to use cargo-args format.
- src/emit/{github,ado}.rs unit tests: assert presence of OX_CHECK_INCLUDE_REQUIRED in the emitted output.
- Snapshots regenerated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Audit of checks.md against cfg(target_os) sensitivity found that several Linux-only groups contain checks whose results can differ across host OS — they compile or rustdoc the crate graph for the host target only, so #[cfg(target_os = ...)] gated code is invisible to a Linux-only run. Compile-sensitive checks identified: - clippy, doc-build, udeps, semver-check, external-types (in pr-fast / nightly-advisories) - mutants (diff and full) — surveyed evidence from oxidizer's pipeline runs mutation testing on both Linux_x64 and Windows_x64 with sharding (oxidizer-github settles for Linux-only as a budget tradeoff). We match oxidizer's policy since it's the stronger signal. - cargo-hack --feature-powerset, cargo bench --no-run (in nightly-exhaustive) — both invoke cargo check/build for permutations. Group OS scope changes: - pr-fast: Linux -> Linux + Windows. Text checks (fmt, license-headers, …) running redundantly on the Windows leg costs <30s vs. the setup overhead of splitting into a separate group. - pr-mutants: Linux -> Linux + Windows. Diff-scoped --in-diff keeps each leg bounded. - nightly-advisories: Linux -> Linux + Windows. Lets clippy and udeps see cfg-gated code on both OSes. - nightly-exhaustive: Linux -> Linux + Windows. Includes mutants-full; adopters who can't afford the Windows mutants leg override the matrix in their own workflow / pipeline (since mutants-full can run for hours). - nightly-runtime, nightly-test, pr-test: unchanged. Template changes: - templates/github/pr-impl-workflow.yml: pr-fast and pr-mutants now strategy.matrix over [linux, windows]. - templates/github/nightly-impl-workflow.yml: nightly-advisories and nightly-exhaustive now matrix over [linux, windows]. - templates/ado/pr-stages.yml: pr_fast and pr_mutants stages each have two jobs (linux + windows) using linuxPool / windowsPool parameters. - templates/ado/nightly-stages.yml: parallel changes for nightly_advisories and nightly_exhaustive. Doc changes: - checks.md section 1: per-group OS scope table updated; per-group reasoning explains the cfg-sensitivity argument and the oxidizer-policy match. - design.md section 8.3: OS-scope table rewritten with the per-group rationale. Note: the linuxPool / windowsPool ADO parameters are NOT toggles for OS inclusion — empty-pool elision was dropped earlier in plan 0001. Their purpose is for adopters to swap in their own pool (e.g., 1ESPT-internal or a self-hosted Microsoft pool) without editing the templates. Snapshots regenerated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two related corrections to the OS-matrix defaults, based on actual upstream evidence: 1. nightly-runtime should not be Linux-only. The earlier 'miri and careful are Linux-primary tooling' framing was a guess. The upstream evidence contradicts it: miri's README explicitly supports all Rust Tier 1 targets (which includes x86_64-pc-windows-msvc) and runs them on every Rust PR. Cross-checking the surveyed repos: - oxidizer (ADO): cargo-miri.yml iterates over each platform in parameters.platforms with no OS filter, so miri runs on Windows_x64 + Linux_x64. - oxidizer-github (GH): miri and careful run in the extended-analysis job matrixed over four legs (Linux/Windows × x86_64/aarch64). So both surveyed repos run nightly-runtime cross-OS. ox-check now matches. 2. GitHub default matrix gains aarch64 legs. oxidizer-github's extended-analysis matrix uses ubuntu-24.04-arm and windows-11-arm (Microsoft-hosted ARM runners that GH ships since 2024-2025). The GH backend's default matrix is now Linux/Windows × x86_64/aarch64 (four legs) across every cross-OS group. Two new workflow inputs (linux_arm_runner default ubuntu-24.04-arm, windows_arm_runner default windows-11-arm) let adopters override the labels. 3. ADO stays two-leg. ADO has no Microsoft-hosted ARM agents (no vmImage exists for Linux aarch64 or Windows aarch64), so the ADO default remains x86_64 Linux + x86_64 Windows. This matches oxidizer's platforms list. Adopters with self-hosted ARM pools extend the stages template in their own root pipeline; ox-check does not reintroduce empty-pool elision (dropped earlier in plan 0001). Implementation: - templates/github/pr-impl-workflow.yml: pr-fast and pr-mutants matrices grow to [linux, windows, linux-arm, windows-arm]; pr-test default test_os grows to 'linux,windows,linux-arm,windows-arm'; two new runner inputs added with sensible defaults; codecov upload still gates on matrix.os == 'linux' (single canonical leg, matches oxidizer-github's policy). - templates/github/nightly-impl-workflow.yml: parallel changes; nightly-runtime, nightly-advisories, nightly-exhaustive all matrix over the four legs. - templates/ado/nightly-stages.yml: nightly-runtime stage gets a windows job alongside the existing linux job. - docs/design/checks.md section 1: OS-scope tables updated; backend-asymmetry note added; nightly-runtime rationale rewritten with the corrected facts. - docs/design/design.md section 8.3: OS-scope rationale rewritten; backend split (GH four-leg vs ADO two-leg) called out as the new opinion. - docs/design/github.md section 4: input table updated with linux_arm_runner / windows_arm_runner; self-hosted-runner example updated. - docs/design/ado.md section 4: ARM coverage gap note added at the top. Snapshots regenerated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two blockers found while reviewing readiness for dogfooding on ox-tools. 1. mod.just had no explicit set shell. checks.just recipes use POSIX shell guards (@if [ ... ]; then ...; fi); on Windows just defaults to cmd /c, which can't parse them — pr-fast / pr-mutants on the Windows leg would fail before any check ran. Fix: set shell := [bash, -cu] in mod.just. GH-hosted and ADO-hosted Windows runners ship Git Bash; local Windows developers near-universally have Git for Windows installed. 2. Cache key was missing the architecture dimension. With the new 4-leg GH matrix x86 and ARM legs of the same OS collide on cache key; target/ is arch-incompatible. Fix: add runner.arch to GH key and Agent.OSArchitecture to ADO Cache@2 key. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two cargo-delta integration bugs found by actually installing the binary and running it:
1. JSON keys are TitleCase (Modified, Affected, Required) — not lowercase. The jq queries in impact-action.yml / impact.yml read .modified / .affected / .required and silently returned empty arrays, which the format_set helper turned into the --skip sentinel. Every PR would therefore execute zero scoped checks. Fix: use the actual key names.
2. There is no --base <ref> flag on cargo delta impact. The real CLI takes two pre-computed snapshots (--baseline <path>, --current <path>). My templates used --base, the command exited 1, the 2>/dev/null swallowed the error, and the JSON fell through to {}. Fix: snapshot both sides explicitly. The baseline snapshot runs inside a temporary git worktree add --detach at the merge target, so the checked-out source tree stays untouched. cargo-delta-current uses the regular working tree.
Tool-minimum versions: surveyed against oxidizer (.pipelines/variables/cargo-tools.yml) and oxidizer-github (constants.env); set each minimum to the lower of the two so we accept either repo's actively-used version. Added cargo-delta=0.3.1 to the catalog so ox-check-tools-install covers the impact-step dependency. Major bumps for cargo-mutants (24.5 -> 26.1.2), cargo-llvm-cov (0.6.13 -> 0.8.4), cargo-deny (0.16 -> 0.19), cargo-ensure-no-default-features (0.1 -> 1.0), cargo-doc2readme (0.6.0 -> 0.6.4), cargo-aprz (0.6 -> 1.0).
Snapshots regenerated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
martin-kolinek
changed the title
While preparing to dogfood on ox-tools, found that the existing top-level
justfile already declares set windows-shell := [pwsh.exe, ...]. just's
windows-shell takes precedence over set shell on Windows, so the
previous global set shell := [bash, -cu] in mod.just would have been
overridden — every guarded recipe in checks.just would still hit pwsh
on Windows CI legs and break on the POSIX-shell syntax.
Fix:
- Drop set shell from mod.just. Adopters keep their own shell
preferences for their own recipes.
- Annotate every checks.just recipe whose body uses POSIX shell syntax
(test, parameter expansion, multi-line conditionals) with
[script("bash")]. just's per-recipe script attribute overrides
shell / windows-shell / script-interpreter unconditionally, so the
recipe runs in bash regardless of how the adopter has configured
the rest of their justfile.
- Single-command recipes (cargo deny check, cargo audit, cargo aprz
check, cargo mutants --workspace ...) inherit the adopter's default
shell — they're shell-agnostic.
- Recipe bodies that previously used @if [ ... ]; then ...; fi; <cmd>
with backslash continuations are rewritten as multi-line script
bodies (no continuations needed inside a [script] recipe).
- The header comment in checks.just documents the policy so future
contributors don't reintroduce the intrusive set shell.
Snapshots regenerated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Initial bootstrap of cargo-ox-check against the ox-tools workspace. `cargo ox-check update` ran cleanly with the GitHub backend autodetected from origin. The 28 emitted items are: - `justfiles/ox-check/` recipe tree (mod / tools / tool-minimums / checks / groups / tiers). Owned files; future edits go through `cargo ox-check update`. - `Justfile` managed region splicing in the ox-check imports alongside the existing hand-written imports — no user content touched, only the region appended at the tail. - `Cargo.toml [ox-check-workspace-lints]` managed region with the opinionated workspace lint catalog. - Per-member `[ox-check-lints]` regions on each of the four workspace crates (automation, cargo_ensure_no_cyclic_deps, cargo_heather, cargo_ox_check). - `deny.toml` / `rustfmt.toml` / `.delta.toml` managed regions. - `.github/actions/ox-check-*` composite actions (setup, impact, pr-fast, pr-test, pr-mutants, nightly-test, nightly-advisories, nightly-runtime, nightly-exhaustive). - `.github/workflows/ox-check-pr.yml` + `ox-check-pr-impl.yml` + `ox-check-nightly.yml` + `ox-check-nightly-impl.yml`. - `.ox-check.lock` sidecar manifest tracking checksums. The existing hand-written `main.yml` and `nightly.yml` are kept in place for this PR — they gate the merge of cargo-ox-check itself and shouldn't be retired until ox-check-emitted CI proves green. Retirement is a follow-up. The hand-written `regenerate-check.yml` (the one bootstrap workflow) is unchanged; once this commit lands its dry-run gate turns green. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
set unstable is required by `[script(...)]` annotations, but declaring it in mod.just conflicts when the adopter's root justfile already has it (just rejects duplicate `set unstable` across the import tree). ox-tools' justfile sets it at the top; mod.just used to redundantly re-set it, blocking `just ox-check-*` from running. Drop the declaration from mod.just and document that the adopter is responsible for ensuring `set unstable` is present in their root justfile (or in any of its imports). For a fresh adopter, `cargo ox-check update` should ideally splice it into the managed region in the adopter's justfile — TODO follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
While dogfooding on this repo, three issues with the previous
`[script("bash")]` approach surfaced:
1. `[script(...)]` is an unstable just feature, requiring
`set unstable`. When the adopter's root justfile already has
`set unstable` (ox-tools' does), the imported mod.just couldn't
redeclare it — error: "Setting `unstable` first set on line 5
is redefined on line 11". Workaround was to drop `set unstable`
from mod.just, which then broke fresh adopters who didn't have
it elsewhere.
2. On Windows, just invokes the `[script("bash")]` interpreter with
the recipe's temp-file path, and bash's argument parser interprets
backslashes in the Windows path as escapes, producing a mangled
path like `C:UsersmakolnekAppDataLocalTempjust-...`. just gives
no knob to fix this, and it's the same with WSL bash, Git Bash,
or any other bash on PATH.
3. Schema test `just_lists_emitted_recipes` failed in the bare
workspace tempdir for the same `set unstable` reason as (1).
Fix: rewrite every recipe whose body used `[script("bash")]` or
`[script("pwsh")]` as a shebang-style recipe (`#!/usr/bin/env
bash` or `#!/usr/bin/env pwsh` as the first body line). Shebang
recipes:
- Don't need the unstable attribute → no conflict with adopter's
existing `set unstable`.
- On Unix, dispatch via the normal shebang mechanism.
- On Windows, just translates the temp-file path with `cygpath`
(shipped with Git for Windows), so bash receives a POSIX path that
it can actually open. Tested locally on Windows with just 1.51 +
Git Bash on PATH.
Single-command recipes (`cargo deny check`, `cargo audit`, etc.)
stay as plain just recipes and inherit the adopter's default shell.
They're shell-agnostic; the shebang ceremony isn't warranted.
Header comments in checks.just / tools.just / mod.just rewritten to
document the new policy. `JUST_UNSTABLE` env var hack in
`schemas.rs` reverted since shebang recipes don't need it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Real-repo dogfood surfaced three classes of duplicate-key conflicts
between the existing hand-written ox-tools config and the regions ox-check
splices in. Resolved per the dirty-file flow design: customizations
live inside the managed region and the LeaveAlone decision preserves
them across template bumps.
1. `Cargo.toml` workspace lints. The hand-written
`[workspace.lints.rust]`, `[workspace.lints.clippy]`, and
`[workspace.lints.rustdoc]` blocks duplicated keys with ox-check's
`[ox-check-workspace-lints]` region. TOML rejects re-opening a
table that's already been declared. Resolved by:
- Removing the hand-written `[workspace.lints.*]` blocks entirely.
ox-check's catalog now owns the workspace lints.
- Moving ox-tools-specific extras (rust.unexpected_cfgs check-cfg
for our coverage attrs; 15 extra clippy restriction lints;
2 extra allow-overrides) INSIDE the managed region as a
customization block at the bottom. Future `cargo ox-check
update` runs see the region as user-modified and apply
LeaveAlone, preserving the customization. To bring extras
upstream, propose adding them to ox-check's catalog.
2. Per-crate `[lints]` blocks. Each member crate had a hand-written
`[lints]\nworkspace = true` that duplicated ox-check's
`[ox-check-lints]` region of identical content. Removed the
hand-written blocks; ox-check's region carries them.
3. `rustfmt.toml` `max_width`. Hand-written `max_width = 140` at
top of file conflicted with ox-check's region default of 110.
Resolved by replacing the value inside the managed region with 140
(the ox-tools preference) and removing the hand-written copy. The
region is now user-modified and stays LeaveAlone across updates.
The `.ox-check.lock` manifest and `justfiles/ox-check/*` tree are
re-emitted after the shebang-template fix landed in the previous
commit; nothing dogfood-specific, just reflecting the latest binary
output.
Verified locally on Windows with just 1.51 + Git Bash:
- `just ox-check-fmt` runs and reports a real formatting drift on
the existing source (since max_width moved 110→140 across the
workspace). To be fixed in a follow-up reformat commit.
- `just ox-check-deny` correctly invokes `_ox-check-require`
which reports `cargo-deny is at v0.18.5; need >=v0.19.0` with
a clear upgrade hint. Tool-version policy works end-to-end.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The earlier shebang-style (#!/usr/bin/env bash) approach worked but
required `cygpath` on PATH on Windows, which is only available
inside a Git Bash session. From plain PowerShell — the default
Windows developer shell — `just ox-check-*` errored with
`could not find cygpath executable to translate recipe shebang
interpreter path`.
pwsh is the better Windows interpreter:
- Preinstalled on Windows 10+ (no separate install)
- Preinstalled on GH ubuntu-latest, ADO Microsoft-hosted Linux
agents, and the equivalent Windows runners
- Available on macOS / Linux via Homebrew or the upstream
PowerShell installer
- The existing _ox-check-require pwsh recipe already established
pwsh as a tools-floor requirement, so this is consistent
So this commit:
- Rewrites every multi-statement recipe in checks.just from
`#!/usr/bin/env bash` shebang to `[script("pwsh")]` with a
pwsh script body. Single-command recipes (deny, audit, aprz,
mutants-full) stay as plain just recipes that inherit the
adopter's default shell.
- Rewrites tools.just (_ox-check-require, ox-check-tools-check,
ox-check-tools-install) in pwsh. The new _ox-check-require uses
the [version] type for SemVer comparison after stripping any
pre-release suffix, replacing the bash sort -V pipeline. The
install hint for pwsh is now per-OS (brew on macOS, winget on
Windows, upstream URL on Linux).
- Include-args splicing uses pwsh's unary -split operator and
array splat: `\ = -split \;
& cargo clippy @pkg ...`. Each --package element ends up as a
separate argv item.
- mod.just header rewritten to document the new policy and call
out the cygpath / bash path-mangling issues that drove the
switch.
[script(...)] requires `set unstable`, which the adopter's root
justfile must declare. ox-check's mod.just does NOT redeclare it (to
avoid conflicting with adopters who already enable it). For fresh
adopters this remains a TODO: ox-check should detect the missing
`set unstable` and either splice it via the managed region or emit
a clear failure message.
Verified locally on Windows from plain PowerShell:
- `just --list` works (lists every emitted recipe)
- `just ox-check-deny` invokes _ox-check-require which correctly
reports the cargo-deny version drift with a clean error message
- `just ox-check-fmt` runs and surfaces real formatting drift
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two cross-platform bugs surfaced during dogfooding when CI ran on Linux against state authored on Windows: 1. Checksums diverged between OSes. Windows `include_str!` of templates read CRLF bytes under git's autocrlf=true, ox-check hashed those CRLF bytes into the manifest, then git committed the files with LF normalization, and on Linux the binary read LF bytes and computed a different checksum. The three-checksum decision algorithm saw L != T even for unchanged templates, tripping regions into Propose instead of LeaveAlone whenever the user-customized content prevented the disk==template short-circuit (which masked the bug for unedited regions). Fix: normalize CRLF byte pairs to LF before hashing in `checksum_bytes` / `checksum_str`. Bare CR bytes are left alone because they're meaningful content rather than a line- ending convention. Adds seven targeted unit tests covering CRLF/LF/mixed/bare-CR/trailing-CR cases. 2. `JUSTFILE_PATH` hardcoded `"Justfile"` (capital J). The ox-tools repo (and most existing repos) use lowercase `justfile`. On case-insensitive Windows the two names map to the same file, hiding the issue locally. On case-sensitive Linux they're distinct, so ox-check looked for the capital-J variant, didn't find it, and tried to emit a second file alongside the existing lowercase one. Fix: switch the constant to lowercase `"justfile"`, matching just's own documented file lookup order (it tries lowercase first). After both fixes, `cargo ox-check update --dry-run` on Windows reports the customized Cargo.toml/rustfmt.toml regions as LeaveAlone (preserving my customizations) and all other items as Unchanged. The same dry-run on Linux CI should now agree. Stale `Justfile` (capital) manifest entry purged on this run; the new `justfile` (lowercase) entry takes over. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…mt max_width to 140
Per the design hint that LeaveAlone is unavoidable only for in-table
overrides, but should be avoidable for additions and for catalog
choices that match the surveyed-repo opinion:
1. ox-tools' Cargo.toml workspace lints customizations (rust.unexpected_cfgs
check-cfg plus 15 extra clippy restriction lints and 2 allow-overrides)
used to live inside the [ox-check-workspace-lints] managed region and
tripped LeaveAlone. The actual TOML scoping rule is that [workspace.lints]
stays open until the next top-level [...] header, so dotted-key
additions after the closing # <<< marker are still inside the same
table. Moved the # <<< up to immediately after the last ox-check-owned
key. The customizations now cohabit the [workspace.lints] table from
outside the region; future template bumps Write cleanly.
2. rustfmt.toml max_width was an *override* (140 vs ox-check catalog's 110),
which can't cohabit a flat root TOML without duplicate-key error. Survey
of the four repos that have rustfmt.toml:
- oxidizer-github: max_width = 140
- ox-tools-gh: max_width = 140 (previous customization)
- oxidizer / assistants-oxide / ox-docs: not set (rustfmt default 100)
The repos that bothered to opine on width chose 140. ox-check catalog
default bumped from 110 to 140 to match. Adopters who actively prefer
100 (or another width) can override outside the managed region, knowing
it would conflict and force them into a LeaveAlone — for which the
right call is to upstream their preference, not customize.
Dry-run now reports all 28 items as Unchanged. No LeaveAlone anywhere.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When a previously-tracked owned file or managed region is no longer in
the current catalog (because the catalog moved on, the user dropped a
backend, or a workspace member went away), ox-check now actively
removes the orphan rather than silently dropping the manifest entry
and leaving the artifact behind.
Decision rules (parallel to the three-checksum table for active items):
D absent - manifest entry purged silently
D == L - Remove: delete the file or splice the region out
D != L - OrphanedKept: leave the disk content alone,
drop the manifest entry, transfer ownership
Implementation:
- `Decision::Remove` and `Decision::OrphanedKept` variants added to
the enum; both report `writes() == true`. `decide_removal()`
produces them from the L vs D comparison.
- `region::remove_region()` splices the markers + body out of a host
file. The trailing blank line is consumed when the region sits mid-
file (preserving exactly one blank between user content) or the
leading blank is consumed when the region sits at end-of-file
(avoiding a trailing orphan blank).
- `Plan::apply` gains branches for `Remove` and `OrphanedKept`
on both file and region targets. The previous safety-net "purge
every manifest entry not in the live plan" loop is removed — explicit
plan items are the source of truth now.
- `run::plan_removals()` scans the previous manifest for entries
not covered by active plan items, computes the disk checksum (with
line-ending normalization, the same pipeline as the active-item
algorithm), and produces the corresponding plan items.
- `Plan::summary` gains "Will remove" and "Orphaned (customized;
transferring ownership)" sections; the older implicit "Stale manifest
entries" footer is gone.
Tests cover every cell of the decision table at the unit level, the
region splice helper (mid-file and end-of-file cases), and the apply
side (file delete, region splice, orphan preservation, idempotent
deletion of already-missing files).
`docs/design/updates.md` gains section 5.1 documenting the new
orphan-handling table and revises section 9 (dry-run UX) to describe
the new summary categories.
Verified against this repo's dogfooded state:
`cargo ox-check update --dry-run --no-backends` now reports
`Will remove: 1 item(s) - .github/workflows/ox-check-pr.yml` plus
the workflows whose manifest L hasn't been refreshed since the line-
ending-normalization commit (which show as Orphaned). Re-running
`update` with backends enabled refreshes the L values and would
then move them to `Will remove` on a subsequent `--no-backends`
dry-run.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nents # What CI on the dogfooded ox-tools repo failed on every matrix leg before even running a check. Three distinct root causes: - **Linux x86_64 + Linux ARM**: `ox-check-tools-install` failed while compiling `cargo-spellcheck` because its `clang-sys` build script can't find `llvm-config` / `libclang.so`. The GH-hosted Ubuntu images don't ship `libclang-dev` on PATH by default. - **Windows x86_64**: tools installed fine (libclang ships with the Visual Studio install on `windows-*` images), but `ox-check-fmt` failed with `cargo-fmt.exe is not installed for the toolchain`. rustup auto-installs the channel from `rust-toolchain.toml` on first cargo invocation but doesn't pull components (rustfmt, clippy) along with it. - **Windows ARM**: `cargo-mutants 27.0.0` source build fails on `aarch64-pc-windows-msvc` with 285 errors compiling `winapi` (upstream incompatibility). # Fix In `setup-action.yml` (GH composite) and `setup.yml` (ADO step template): - New conditional step: `sudo apt-get install -y libclang-dev` on Linux. macOS Xcode CLT bundles libclang already; Windows VS install has its own LLVM, so no install on those OSes. - New unconditional step: `rustup component add rustfmt clippy`. Cheap on every OS because rustup is idempotent — adds the components if missing, no-op if already present. Both components are catalog-essential (fmt + clippy recipes); leaving them implicit in `rust-toolchain.toml` was the bug. For the Windows ARM cargo-mutants build failure, the right fix is to exclude mutants from the ARM legs entirely. Per the same logic, `pr-mutants` and `nightly-exhaustive` are now x86_64-only across both backends: - `pr-impl-workflow.yml`: pr-mutants matrix narrowed to `[linux, windows]`. - `nightly-impl-workflow.yml`: nightly-exhaustive matrix narrowed to `[linux, windows]`. - `checks.md` and `design.md` §8.3: per-group OS scope tables updated; rationale calls out the `winapi` constraint and the value-per-leg argument for keeping mutation testing on x86_64. Adopters who explicitly need ARM mutation coverage extend the matrix in their root workflow. ADO unchanged (already x86_64-only by default since Microsoft-hosted ADO has no ARM agents). # Dogfood state Snapshots regenerated. Dogfooded state on this repo: `cargo ox-check update --dry-run` reports all 28 items as Unchanged after accepting the three .ox-check-proposed sidecars produced by this commit's template churn (setup-action.yml + pr-impl-workflow.yml + nightly-impl-workflow.yml). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Bug
After landing the line-ending-normalization commit (`6cee738`), all
files continued to report as `Unchanged` even though their manifest
`L` values were computed by the previous (non-normalizing) hash
function. The bug was invisible until a template was edited, at
which point the file would mis-classify as `Propose` (suggesting
user customization) instead of `Write` (clean update).
# Root cause
The decision algorithm checks `F == T` first:
\\\
ust
match (inputs.disk, inputs.last_rendered) {
(None, _) => Decision::Write,
(Some(d), _) if d == inputs.template => Decision::InSync, // <- here
...
}
\\\
When `F == T` (computed with the same binary's hash function on
both sides), this arm fires and `L` is never consulted. A stale
`L` from an older binary version sits in the manifest unread,
unnoticed. The moment `T` moves, the InSync short-circuit no
longer fires; the algorithm falls through to the `L` checks; `F
!= L` (different hash function) AND `T != L` (same) — so
`Propose` instead of `Write`.
# Fix
When `InSync` is observed, opportunistically refresh `L = T` in
the manifest. No file write happens — the disk content already
matches the template. But the manifest entry gets self-healed to
the current binary's hash space, so a subsequent template change
correctly lands as `Write` rather than `Propose`.
Mechanically:
- New `PlanItem::insync(target, template_checksum)` constructor
carries the template checksum; emit code uses it in the InSync
branch instead of the bare `PlanItem::noop` (which doesn't
carry a checksum).
- `Plan::apply` for InSync now refreshes `next.files` /
`next.regions` with the carried checksum if present (the
`LeaveAlone` branch deliberately does NOT refresh, because
bumping `L = T` there would let the next run silently
overwrite the user's customization).
- Two new tests pin both behaviors:
`apply_insync_refreshes_stale_manifest_l` (positive case) and
`apply_leave_alone_does_not_refresh_manifest_l` (negative — must
preserve user customization).
# Verified
After this fix, `cargo ox-check update` on the dogfooded ox-tools
repo refreshes every stale L on the first run (all 28 InSync). A
follow-up `--no-backends` dry-run now correctly reports all 13
github-emitted files as `Will remove` (was 1 Remove + 12 Orphaned
before this fix, because the stale-L was misclassifying them as
"user customized").
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two bugs surfaced in the dogfooded CI run on commit `9e0e619`: # 1. `ox-check-mutants` recipe passed a git revision range to `cargo mutants --in-diff`, but that flag takes a FILE PATH containing a unified diff. The recipe now writes the diff to a temp file first (`\` / `\` / system temp dir fallback) and passes the file path. Symptom from CI: `ERROR Failed to open diff file: No such file or directory (os error 2)`. # 2. `ox-check-tools-install` aborted on the first per-tool install failure, blocking unrelated groups. Concretely: `cargo-mutants 27.0.0` doesn't build on `aarch64-pc-windows-msvc` (upstream `winapi` incompat with 285 errors), so the windows-arm `pr-fast` leg failed at install even though pr-fast doesn't need cargo-mutants at all. Recipe now continues on per-tool failures, collects the failed tools, and exits 1 at the end if any failed (so the failure is visible in CI summary). Recipes that actually USE a missing tool fail cleanly via `_ox-check-require` at recipe-run time; recipes that don't need it proceed unaffected. Setup steps in `setup-action.yml` (GH) and `setup.yml` (ADO) gain `continue-on-error: true` / `continueOnError: true` so the install step's non-zero exit doesn't fail the job. The per-recipe `_ox-check-require` then enforces what each group actually needs. # Verified locally Snapshots regenerated. Dogfood ran cleanly with three Write updates landed (tools.just + checks.just + setup-action.yml) and 25 Unchanged. The recently-landed InSync self-heal (`9e0e619`) is what kept this from showing as Propose — confirming both fixes compose. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The pr-test and nightly-test jobs in the GH backend silently never
ran because their matrix expression couldn't parse the input.
The previous setup combined a CSV-style `test_os` default like
`linux,windows,linux-arm,windows-arm` with a matrix expression of
`fromJSON(format('[{0}]', inputs.test_os))`. The format() call
produced the literal string `[linux,windows,linux-arm,windows-arm]`,
which is NOT valid JSON because the values lack quotes. fromJSON
returned null and the matrix had zero entries, so GH silently
scheduled zero jobs for pr-test / nightly-test. Result: no pr-test
on the dogfooded PR even though tests should have run.
The fix is to make `test_os` already JSON-array-shaped so the matrix
expression can consume it directly:
test_os:
default: '["linux","windows","linux-arm","windows-arm"]'
...
matrix:
os: fromJSON(inputs.test_os)
Adopters who override now pass JSON arrays:
- Drop ARM legs:
with: { test_os: '["linux","windows"]' }
- Add macOS:
with: { test_os: '["linux","windows","linux-arm","windows-arm","macos"]' }
Applied to both pr-impl-workflow.yml (pr-test matrix) and
nightly-impl-workflow.yml (nightly-test matrix). Docs in github.md
updated for the new input format.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…uts.X)
Workflow templates previously mixed two ways of providing the OS
matrix:
- Hardcoded inline arrays:
matrix:
os: [linux, windows, linux-arm, windows-arm]
- Input-driven via JSON parse:
matrix:
os: fromJSON(format('[{0}]', inputs.test_os)) # buggy
Only pr-test and nightly-test used the input-driven pattern, and
the format() / fromJSON() combination was broken — the previous
test_os default was a CSV string ("linux,windows,...") which
format() wrapped to "[linux,windows,...]", which is invalid JSON
because the values lack quotes. fromJSON returned null and the
matrix expanded to zero entries, silently skipping the entire job.
Two-step refactor:
1. The previous commit changed test_os to a JSON-array string
(default '["linux","windows","linux-arm","windows-arm"]') and
dropped the format() wrapper so fromJSON could consume the
input directly.
2. This commit extends that pattern uniformly. Every multi-OS
group's matrix is now driven by an input — no hardcoded inline
arrays remain. The input set is two values:
- `os` (default 4 legs): drives pr-fast, pr-test,
nightly-test, nightly-advisories,
nightly-runtime
- `mutants_os` (default 2 legs): drives pr-mutants,
nightly-exhaustive
(x86_64 only because
cargo-mutants doesn't build on
aarch64-pc-windows-msvc)
The two-input split (rather than one global `os`) reflects a real
constraint: cargo-mutants compatibility on ARM Windows. Folding
both groups under a single `os` would force adopters to either
include broken legs (failing) or drop ARM coverage for the OTHER
groups too (regression).
ADO is already consistent — every multi-OS stage uses explicit
`- job: linux` + `- job: windows` blocks via parameters.linuxPool /
windowsPool. No ADO change.
Templates:
- pr-impl-workflow.yml: pr-fast and pr-test now use
fromJSON(inputs.os); pr-mutants uses fromJSON(inputs.mutants_os).
No more hardcoded matrices. Inputs renamed from `test_os` to `os`
and a new `mutants_os` added.
- nightly-impl-workflow.yml: same pattern for nightly-test,
nightly-advisories, nightly-runtime (all use `os`); nightly-
exhaustive uses `mutants_os`.
- github.md inputs table updated. The unit test that previously
asserted PR_IMPL_WORKFLOW contains "test_os" now asserts both
fromJSON(inputs.os) and fromJSON(inputs.mutants_os).
Snapshots regenerated. Dogfooded re-emit landed cleanly as Write
(no Propose) thanks to the InSync self-heal landed earlier.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert input-driven matrices (os / mutants_os as JSON-array workflow_call inputs consumed via romJSON) in favor of hardcoded inline YAML arrays in the owned reusable workflows. Per-leg runner *labels* remain inputs so adopters can swap in self-hosted runners, but the OS axis shape itself is now part of the workflow's identity. Adopters who need a different shape (add macOS, drop ARM, mix in exotic targets) fork the emitted impl workflow and dirty-file flow takes over. Rationale: input-driven matrices added a silent failure mode (a mis-formatted CSV/JSON input silently produces an empty matrix that GitHub Actions treats as o legs to run), and the surveyed repos (oxidizer-github, oxidizer) all use hardcoded matrices. The kinds of adopters who want to change the OS axis are almost certainly making other changes too, so requiring a fork isn't a meaningful escalation. Updates: pr-impl-workflow.yml, nightly-impl-workflow.yml (templates); emit/github.rs assertions; github_backend snapshot; design.md / github.md / checks.md docs to match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Picks up template change from previous commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add steps/job.yml — a per-job wrapper with a small, stable parameter contract (name, pool, steps, artifacts). Every job in pr-stages.yml and nightly-stages.yml is now rendered through this wrapper. Adopters whose ADO instance requires extension templates (1ES PT, SubstratePT, M365PT, custom corporate templates) take ownership of steps/job.yml — adding templateContext: blocks, build-provenance attrs, SDL hooks, etc. — and ox-check stops overwriting it via the standard dirty-file flow. The stages templates continue to update normally because they only know about the wrapper's parameter contract, not its body. Why a dedicated wrapper rather than parameterizing the stages template: the wrapper is six lines and stable; the stages template is long and changes often (new groups, new dependsOn wiring, new impact-output threads). Putting the user's customization in a separate file means stages updates flow through without merge conflicts, and the wrapper customization survives every ox-check upgrade. Why owned-but-customizable rather than proposed-once: first-time adoption needs zero extra steps — a fresh \cargo ox-check update\ writes a working wrapper and the pipeline runs. Dirty-file behavior kicks in only when the user actually edits it. The artifacts: parameter bridges the vanilla-vs-1ESPT asymmetry: vanilla ADO uses PublishPipelineArtifact@1 tasks (allowed at step level), 1ESPT requires templateContext.outputs.pipelineArtifact at job level. The stages templates declare artifacts uniformly; the wrapper translates per backend. Code coverage stays as a sibling task (PublishCodeCoverageResults is allowed at step level in both vanilla and 1ESPT). Template-path note: stepList parameters preserve template-path resolution relative to the call site (the stages template), so paths like 'steps/pr-fast.yml' continue to work unchanged. Changes: - New: templates/ado/steps/job.yml - Refactored: templates/ado/pr-stages.yml, nightly-stages.yml — every job now delegates through steps/job.yml - Added JOB_WRAPPER constant + plan registration in src/emit/ado.rs - New test fn job_wrapper_declares_expected_contract - Updated assertions in pr_stages_has_impact_and_group_stages and nightly_stages_has_four_groups to require wrapper delegation and forbid bare \- job:\ keys - Updated ado_backend snapshot - docs/design/ado.md: rewrote §4 around the wrapper (new §4.1), updated §2 emitted-artifacts list Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces
cargo-ox-check: a cargo subcommand that ships and maintains an opinionated, unified build/CI scaffolding for Rust workspaces. Onecargo ox-check updateinvocation emits a complete, reviewable tree of GitHub Actions workflows, Azure DevOps pipelines,justfiles/ox-check/recipes, and managed regions inCargo.toml/deny.toml/rustfmt.toml/.delta.toml. Subsequent runs keep the tree in sync as the tool's catalog evolves, while preserving any local customizations the adopter has made.Why this exists
Today every Rust repo in the
oxidizer/oxidizer-github/ox-toolsfamily hand-rolls its own CI on the same opinionated checks (fmt, clippy, deny, audit, miri, careful, mutants, llvm-cov, cargo-hack, semver-check, doc, ...). The cost of keeping those copies in sync — toolchain bumps, lint updates, new advisory feeds, cross-OS matrix changes — is paid per-repo, by hand, every time. cargo-ox-check centralizes the catalog in one place and renders it into the per-repo CI surface.What an adopter gets
After one
cargo ox-check update, the repo contains:.github/workflows/ox-check-pr.yml/ox-check-pr-impl.yml+ composite actions for the PR tier (pr-fast,pr-test,pr-mutants)..github/workflows/ox-check-nightly.yml/ox-check-nightly-impl.yml+ composite actions for the nightly tier (nightly-test,nightly-advisories,nightly-runtime,nightly-exhaustive)..pipelines/ox-check/mirror of the same wiring for Azure DevOps.justfiles/ox-check/recipe tree (checks.just,groups.just,tiers.just,tools.just,mod.just,tool-minimums.txt) — localjust ox-check-prreproduces the PR CI without any ox-check binary present.Cargo.toml(opinionated workspace + per-crate lints),rustfmt.toml,deny.toml,.delta.toml..ox-check.lockmanifest that tracks per-file/per-region checksums for the three-checksum update algorithm.Design highlights
modified,affected,required) backed bycargo-delta. Each check is tagged with the tier it scopes to; the CI wiring emits one include-list env var per tier with a--skipsentinel for empty tiers. The required tier is included for tools whose correctness resolves through the dep graph (cargo doc,cargo hack,cargo udeps); the unscoped bucket is reserved forCargo.lock- and PR-context-only checks (deny,audit,aprz,pr-title).Write,LeaveAlone,Propose) keyed on a three-way comparison of (original-template, current-template, live-file) checksums, all tracked in the lock manifest. A user edit inside a managed region is preserved when the template hasn't moved; a template bump becomes a.ox-check-proposedsidecar with a one-line summary, so customizations are never silently overwritten.origin's URL; explicit--backend github/--backend adooverride.actions/cache, ADOCache@2) keyed on OS + arch + rustc version + lockfile-family hashes +tool-minimums.txthash, so toolchain bumps and catalog updates invalidate cleanly..github/workflows/regenerate-check.yml— the one hand-written workflow that buildscargo-ox-checkfrom the PR's branch, runscargo ox-check update --dry-run, and fails the PR if the in-tree state drifts from what the templates would render.Verification
crates/cargo_ox_check/src/(run/plan/decision/region/emit/workspace/manifest/checksum).tests/schemas.rs(every emitted TOML parseable; manifest schema valid).tests/snapshots.rscovering the three representative backend combinations (local-only, GitHub-backend, ADO-backend) — full byte-exact emitted tree.tests/update.rscovering single-crate (no[workspace]), opt-outs (emptied managed region preserved), customized (user edit inside a managed region preserved), migration (pre-existing hand-writtenJustfile/deny.toml/[profile.release]survive splicing).cargo clippy -p cargo-ox-check --all-targets -- -D warningsclean.cargo-deltaCLI and JSON shape verified by running the actual binary against this workspace; the impact-step shell scripts encode the real two-snapshot--baseline/--currentflow.Total: 168 tests green locally.
Documentation
Lives under
crates/cargo_ox_check/docs/:design/design.md— top-level design and CLI shape.design/local.md— thejustrecipe tree, impact env vars, daily-driver flow.design/github.md— owned reusable workflows, per-group composite actions, impact scoping.design/ado.md— owned stages templates, per-group step templates, 1ESPT composition guidance.design/checks.md— the opinionated catalog and per-group OS scope matrix.design/updates.md— the three-checksum state machine validated by fixture tests.verification.md— continuous-validation strategy (dogfooding + snapshot + fixtures + schema).implementation-plans/0000.md— initial implementation plan (history).implementation-plans/0001.md— design-vs-implementation reconciliation plan executed in this PR.Follow-ups (not in this PR)
cargo ox-check updateagainstox-toolsto land the emitted CI surface (the regenerate-check workflow will start passing the moment the maintainer commits that bootstrap output).pr-titleConventional Commits regex hasn't been exhaustively validated against edge cases (scoped, breaking-change, mixed-case).