feat(cargo-coverage-gate): add cargo subcommand for per-crate coverage gating by martin-kolinek · Pull Request #32 · microsoft/ox-tools

martin-kolinek · 2026-05-26T17:31:21Z

Adds a new cargo-coverage-gate crate to the workspace: a cargo subcommand that gates pull requests on per-crate line coverage measured by cargo-llvm-cov. The full design and implementation plan live under crates/cargo-coverage-gate/docs/.

What it does

cargo coverage-gate  [--json <path>] [--crates <name>,<name>,...]
                     [--summary-file <path>] [--quiet]

For each gated workspace member, the tool:

Resolves a threshold from [package.metadata.coverage-gate] min-lines → [workspace.metadata.coverage-gate] min-lines → the built-in default of 100.0. Setting min-lines = 0.0 is the explicit opt-out.
Computes per-crate line totals from a cargo-llvm-cov JSON v2 report, attributing files to crates by longest-prefix match on the manifest directory.
Renders a text table to stdout and (optionally) a Markdown table to --summary-file, with auto-detection of $GITHUB_STEP_SUMMARY and $COVERAGE_GATE_SUMMARY.
Exits 0 if every gated crate meets its threshold, 1 if any falls below, and 2 for configuration errors (unparseable JSON, missing file, missing data for a gated crate, unknown name in --crates, out-of-range min-lines, …).

What it is not

It does not run tests or produce coverage data — that is cargo-llvm-covs job.
It does not upload to Codecov or ADO coverage UIs.
It does not write to Cargo.toml; every threshold change is a hand-edited, code-reviewed PR diff.

Design highlights

Per-crate, not diff or workspace-total — same denominator across full and cargo-delta runs, and the displaced-coverage case (high-coverage code replaced with lower-coverage code) is caught because the per-crate number drops.
Loud failure on missing data — a gated crate with zero attributed coverage files is exit 2, not silently OK. Catches typos in --crates, broken impact tooling, and missing test runs.
Cross-crate attribution is documented as a contract on the impact tool (reverse-dependency closure required). See docs/design/main.md §6.4.

Implementation

Eight phases, one commit each:

c3c3501 — Skeleton (Cargo.toml, lib, bin, CLI).
15c45bd — JSON model with lenient version handling.
0c9be5b — Workspace discovery via cargo_metadata; range-validated min-lines.
dc5737c — Three-layer threshold resolution with ThresholdSource tag.
eb84cbb — Attribution, aggregation, verdict — including the §6.3 loud-failure rule.
c595a0f — Text and Markdown renderers; public evaluate / EvaluatedReport API.
b70fb26 — Twelve end-to-end CLI tests covering exit codes, scope filtering, summary files, env-var fallback, malformed input, default-100% behaviour.
c8fde9e — Module docs, README regeneration, .spelling additions.

Test summary

50 library unit tests
1 binary unit test (clap surface)
12 integration tests against the built binary in temp workspaces
1 doctest (library usage example)

cargo build, cargo clippy --all-targets --all-features, and cargo test all pass for the new crate.

Marked as draft

Filed as draft because the surrounding workspace machinery has a few touchpoints that may need a maintainer pass before merge:

just spellcheck could not be executed locally (cargo-spellcheck not installed in the build environment); I added likely-tripping vocabulary to .spelling preemptively but a real spellcheck pass is worth confirming.
cargo-workspaces was not available either, so the README was regenerated by running cargo doc2readme --lib --template ../README.j2 directly inside the crate rather than via just readme. The output is byte-equivalent.
The crate is not yet dogfooded on ox-tools itself; that is a deliberate follow-up after maintainers decide what min-lines values they want to commit.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

New standalone tool for per-crate coverage gating, separate from cargo-ox-check. Reads cargo-llvm-cov JSON output, attributes files to workspace members by longest-prefix match, aggregates per-crate line coverage, and compares against thresholds stored in each crate's [package.metadata.coverage-gate] section. Three subcommands: check (gate), bump (ratchet up thresholds), init (seed thresholds from observed coverage). The check command writes a Markdown verdict table to GITHUB_STEP_SUMMARY (GH) or a configured file (ADO via task.uploadsummary), and exits non-zero when any in-scope crate is below its threshold. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…to 100% - Remove the `bump` subcommand. Every threshold change is now a manual `Cargo.toml` edit, visible in the PR diff and reviewed alongside the code that motivates it. - Add `[workspace.metadata.coverage-gate]` as a workspace-wide default that per-crate metadata overrides. Built-in fallback is `min-lines = 100.0` — gating is on by default; opt out per crate with `min-lines = 0.0`. - Remove references to the unreleased `cargo-ox-check` crate and the `OX_CHECK_IMPACTED` env var; describe the impacted-crate list generically. - Verdict table grows a `Source` column (crate / workspace / default) so the resolved threshold layer is visible at a glance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add a new section 6.4 explaining how per-crate aggregation (grouped by source ownership) interacts with impact-scoped test execution (grouped by test binary), and the contract the gate places on the impact tool: for every crate in the impacted set, include the full reverse-dependency closure so that every test binary capable of exercising the crate runs. Update 6.3 so that a crate in the gated set with zero attributed files in the coverage JSON is a configuration error (exit 2), turning silent gaps into loud failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…0000 - Move docs/design.md to docs/design/0000.md to make room for further design notes alongside the v1 design. - Add docs/implementation-plans/0000.md tracking the phased v1 build: crate skeleton, JSON model, workspace discovery, threshold resolution, attribution + aggregation + verdict, init subcommand, renderers, E2E CLI tests, and docs/release prep. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The tool is now read-only. Threshold values are only ever set by hand in Cargo.toml; with no per-crate or workspace metadata, the gate requires 100% line coverage by default. This removes another mutable code path and keeps every policy change reviewable in a PR diff. Design: simplify section 5.2 to a single check subcommand; drop the init flag descriptions; rewrite section 7.1 adoption around the default-100% behaviour; update section 10.2 to state the tool never writes Cargo.toml. Implementation plan: collapse phases 6 and 7 by removing the init phase, drop init.rs from the crate layout, drop toml_edit from the dependency list, and remove the related risks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The tool now exposes a single command — invocation is just `cargo coverage-gate [flags]`, no nested subcommands. CLI surface shrinks accordingly; the cargo subcommand binary still skips the `coverage-gate` token that cargo prepends to argv when invoked as a subcommand. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Bring up the cargo-coverage-gate crate skeleton: * New crate at crates/cargo-coverage-gate with cargo subcommand binary, library entry point, and error type — all stubs. * CLI surface: cargo coverage-gate [--json] [--crates] [--summary-file] [--quiet]. The single-variant CargoCli enum handles cargo's prepended subcommand-name token. * Public library API: Verdict enum (Pass/Fail/ConfigError -> exit 0/1/2) and a placeholder run() that returns CoverageGateError::NotImplemented. * Add the crate to the workspace dependency list and to the top-level README. cargo build, cargo clippy and cargo test all pass for the new crate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add src/llvm_cov.rs modelling the subset of the cargo-llvm-cov JSON v2 schema the gate actually consumes: report -> data[*].files[*] -> filename and summary.lines.{count,covered}. The functions / regions blocks are accepted by the deserializer but ignored. Version handling is lenient by design. A missing version field or a major version other than 2 emits a tracing warning but continues; only structural parse failures are hard errors (CoverageGateError::JsonParse). Fixtures under tests/fixtures/llvm_cov cover: empty data, single file, multi-file, no-version, unknown-version, malformed JSON, and structurally invalid JSON. Seven unit tests pin the parser. The deserialized types remain crate-private and are wired up in phase 3+; until then they're guarded by cfg_attr(not(test), expect(dead_code)). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add src/workspace.rs wrapping cargo_metadata::MetadataCommand to: * Discover the workspace root (from CWD or an explicit manifest path). * Enumerate workspace members, sorted by name, with each member's manifest directory recorded as an absolute path. * Read the optional [package.metadata.coverage-gate] min-lines value per member and the optional [workspace.metadata.coverage-gate] default. * Validate that every min-lines value falls in 0.0..=100.0, accepting either integer or float TOML representations; out-of-range values produce CoverageGateError::InvalidThreshold, non-numeric values produce CoverageGateError::Metadata. cargo_metadata 0.23.1 added to [workspace.dependencies]; --no-deps is used to skip resolution. Six unit tests cover: no metadata, workspace-level default, per-crate override, out-of-range per-crate, negative workspace, non-numeric. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add src/threshold.rs with the three-layer resolution rule from the design: per-crate then workspace then built-in 100.0. The resolved Threshold carries both the value and a ThresholdSource tag (Crate / Workspace / Default) so the verdict table can show which layer the number came from. Five unit tests pin: per-crate wins, workspace used when no per-crate, default used when nothing set, per-crate 0.0 is honoured as an opt-out rather than treated as missing, and the source labels match the verdict-table strings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Three new modules wire the gate end-to-end: * attribute.rs — longest-prefix path match maps each coverage entry to a workspace member; component-aware so alpha-extras is not folded into alpha; unmatched files are reported separately for an aggregated warning. * aggregate.rs — deterministic per-crate line totals: input files are sorted by path before summing, so two runs over the same data yield byte-identical numbers regardless of input order. * verdict.rs — full Report assembly with one CrateOutcome per gated crate; Status is Ok / Fail / NoData; the overall Verdict obeys the precedence NoData > Fail > Ok. --crates restricts the gated subset and unknown crate names error out. The float comparison uses a 1e-6 epsilon to absorb representation jitter. lib::run is now an end-to-end driver: it parses the JSON, loads the workspace, evaluates the report, and returns a Verdict. The binary loads the JSON file from disk and calls lib::run, mapping Verdict::{Pass,Fail,ConfigError} to exit 0/1/2. Fields and methods that only Phase 6's renderer will display are tagged with cfg_attr(not(test), expect(dead_code)) so the lib build stays warning-free; the unit tests already exercise them. Eighteen new unit tests pin the modules; total is now 42 tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Two output flavours sharing the same Report: * render/text.rs — fixed-width plain-text table matching design §5.4, with auto-sized columns, ─-bar separators, and a one-line summary. * render/markdown.rs — GitHub-flavored Markdown table matching design §6.5, with emoji status indicators and a bold result line. Suitable for GITHUB_STEP_SUMMARY and ADO uploadsummary. * render/mod.rs — shared cell formatters (lines / threshold / delta / status / source) to keep both flavours byte-stable and consistent. Public API revised: the library now exposes evaluate() returning an EvaluatedReport with render_text(), render_markdown(), and verdict() methods. The old run() shortcut is gone — the binary calls evaluate() directly and orchestrates rendering itself. Binary updates: * Always render text to stdout unless --quiet. * Resolve the summary destination: --summary-file > GITHUB_STEP_SUMMARY > COVERAGE_GATE_SUMMARY env var > nothing. * Exit with verdict.exit_code(). Eight new unit tests pin the renderers; phase-5 dead_code suppressions are removed because phase-6 consumers now use those items. Total tests is now 51. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add tests/cli.rs with twelve assert_cmd scenarios that exercise the binary against self-contained tempdir workspaces and synthesized JSON fixtures: * all_pass_mixed_sources — crate + workspace source labels appear. * one_crate_below_threshold_exits_1 — exit 1 plus FAIL row. * gated_crate_with_no_data_exits_2 — exit 2 plus NO DATA row. * crates_flag_restricts_scope — out-of-scope failing crate is hidden. * crates_flag_with_unknown_name_exits_2. * summary_file_flag_writes_markdown — file contents sanity-checked. * github_step_summary_env_is_auto_detected — fallback wiring works. * quiet_suppresses_stdout_but_still_writes_summary. * unknown_version_string_warns_but_continues — exit 0. * malformed_json_exits_2. * missing_json_file_exits_2. * default_threshold_is_100_when_nothing_configured. Binary error handling rewritten for the right exit codes: main now returns std::process::ExitCode, and run() returns Result<ExitCode>. Every library-side failure (JsonParse / Metadata / InvalidThreshold / missing file) maps to exit 2 — the loud-failure semantics from §6.3 of the design. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Round out v1 with publishable documentation: * Enrich lib.rs module docs with a Binary usage section (CLI surface plus exit-code semantics and summary-file fallback), a runnable no_run doctest demonstrating the library API, an explicit note that min-lines = 0.0 is the opt-out, and links to the implementation plan alongside the design doc. * Regenerate README.md via cargo doc2readme so it tracks the lib.rs docs. * Add likely-tripping vocabulary to .spelling: cargo-llvm-cov, llvm, llvm-cov, llvm-tools, Markdown, Codecov, GFM, ADO, lcov, cobertura, nextest, deserializer, deserialization, freeform, prefixed, unparseable, monorepo, kebab, ratchet, subcommand, subcommands. All 64 tests (50 unit + 1 binary + 12 integration + 1 doctest) pass and cargo clippy is clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-05-26T17:32:23Z

⚠️ Breaking Changes Detected

error: failed to retrieve local crate data from git revision

Caused by:
    0: failed to retrieve manifest file from git revision source
    1: possibly due to errors: [
         failed to parse /home/runner/work/ox-tools/ox-tools/target/semver-checks/git-origin_main/9a1e8411c8570e484232187531e7e390fd62d7cb/crates/cargo_ensure_no_cyclic_deps/tests/fixtures/with_cycle/Cargo.toml: no `package` table,
         failed to parse /home/runner/work/ox-tools/ox-tools/target/semver-checks/git-origin_main/9a1e8411c8570e484232187531e7e390fd62d7cb/crates/cargo_ensure_no_cyclic_deps/tests/fixtures/with_self_dev_dep/Cargo.toml: no `package` table,
         failed when reading /home/runner/work/ox-tools/ox-tools/target/semver-checks/git-origin_main/9a1e8411c8570e484232187531e7e390fd62d7cb/scripts/crate-template/Cargo.toml: TOML parse error at line 9, column 26
         |
       9 | keywords = ["oxidizer", {{CRATE_KEYWORDS}}]
         |                          ^
       missing key for inline table element, expected key
       : TOML parse error at line 9, column 26
         |
       9 | keywords = ["oxidizer", {{CRATE_KEYWORDS}}]
         |                          ^
       missing key for inline table element, expected key
       ,
         failed to parse /home/runner/work/ox-tools/ox-tools/target/semver-checks/git-origin_main/9a1e8411c8570e484232187531e7e390fd62d7cb/crates/cargo_ensure_no_cyclic_deps/tests/fixtures/without_cycle/Cargo.toml: no `package` table,
         failed to parse /home/runner/work/ox-tools/ox-tools/target/semver-checks/git-origin_main/9a1e8411c8570e484232187531e7e390fd62d7cb/Cargo.toml: no `package` table,
         failed to parse /home/runner/work/ox-tools/ox-tools/target/semver-checks/git-origin_main/9a1e8411c8570e484232187531e7e390fd62d7cb/crates/cargo_ensure_no_cyclic_deps/tests/fixtures/with_dev_cycle/Cargo.toml: no `package` table,
       ]
    2: package `cargo-coverage-gate` not found in /home/runner/work/ox-tools/ox-tools/target/semver-checks/git-origin_main/9a1e8411c8570e484232187531e7e390fd62d7cb

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: cargo_semver_checks::rustdoc_gen::RustdocFromProjectRoot::get_crate_source
   2: cargo_semver_checks::rustdoc_gen::StatefulRustdocGenerator<cargo_semver_checks::rustdoc_gen::CoupledState>::prepare_generator
   3: cargo_semver_checks::Check::check_release::{{closure}}
   4: cargo_semver_checks::Check::check_release
   5: cargo_semver_checks::exit_on_error
   6: cargo_semver_checks::main
   7: std::sys::backtrace::__rust_begin_short_backtrace
   8: main
   9: <unknown>
  10: __libc_start_main
  11: _start

If the breaking changes are intentional then everything is fine - this message is merely informative.

Remember to apply a version number bump with the correct severity when publishing a version with breaking changes (1.x.x -> 2.x.x or 0.1.x -> 0.2.x).

* spell-check: .spelling was reduced to a single line by a prior PowerShell `Set-Content -NoNewline` mistake. Rebuild it as one word per line, including the new vocabulary added in phase 8. * static-analysis: run `cargo fmt -p cargo-coverage-gate` so the whole crate matches the workspace `rustfmt` settings (notably `max_width = 140`). * coverage: the binary's `cli.rs` carried `#[cfg_attr(coverage_nightly, coverage(off))]` on its test module, but `main.rs` doesn't enable `feature(coverage_attribute)`. Drop the attribute on the binary side (libraries still use it). * mutation-testing: four mutants slipped past phase-7 tests. - Refactor `CoverageReport::warn_on_unsupported_version` into a pure `version_status() -> VersionStatus` query, called from `from_str`. Three new tests pin Supported / Missing / Unsupported. The mutant on the now-removed helper goes away. - Add an explicit assertion in the text renderer test that two ─-bar separator lines appear, killing the `write_separator -> Ok(())` mutant. - Add two tests on `EvaluatedReport::unattributed_count` covering zero and non-zero counts. Phase totals: 54 unit + 1 binary + 12 integration + 1 doctest = 68. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

codecov-commenter · 2026-05-26T18:32:18Z

Codecov Report

❌ Patch coverage is 98.82904% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.1%. Comparing base (17e35dc) to head (bd5dc9d).

Files with missing lines	Patch %	Lines
crates/cargo-coverage-gate/src/verdict.rs	97.1%	3 Missing ⚠️
crates/cargo-coverage-gate/src/render/markdown.rs	91.6%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main     #32     +/-   ##
=======================================
+ Coverage   90.0%   93.1%   +3.0%     
=======================================
  Files         15      28     +13     
  Lines        808    1235    +427     
=======================================
+ Hits         728    1150    +422     
- Misses        80      85      +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

* spell-check: rebuild .spelling from origin/main verbatim and append the new vocabulary; the earlier rebuild via Sort-Object dropped/ re-ordered entries in a way that confused Hunspell on unchanged cargo-heather files. Replace British spellings (modelled, flavours, summarised) with American equivalents; reword 'renderable' and 'JSON''s stored' to avoid possessive/-able tokens; add the remaining technical terms (renderer/renderers, representable, emoji, vs, workspace''s) to .spelling. * extended-analysis (udeps): drop the unused `toml` dependency from cargo-coverage-gate. cargo_metadata exposes Cargo.toml metadata as serde_json::Value, so we never parse TOML directly. * extended-analysis (miri): mark workspace::tests::* and tests/cli.rs tests with `#[cfg_attr(miri, ignore = ...)]` because they spawn the `cargo metadata` subprocess / the binary under test, neither of which miri can execute. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add `v2` and `§` to .spelling — both appear in module docs as references to the cargo-llvm-cov JSON schema version and to design- doc sections respectively. * Reword the summary-target priority comment in run.rs so the `>` separators (which Hunspell flags as words) are gone; the new prose is also more readable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Two leftover spell-check failures from the previous pass were caused by tokenization quirks: * Hunspell tokenizes `§6.5` as `§6` followed by `.5`, so adding `§` to .spelling didn't help. Reword the design-doc reference in src/render/markdown.rs to `section 6.5`. * `env var` was flagged on the abbreviation `env`. Expand to `environment variable` in the summary-target doc comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

* aggregate.rs: drop the path-sort, which did nothing useful — integer addition is commutative and associative, so the order in which we walk the input files doesn't affect the totals. Update the design doc accordingly. The `deterministic_across_input_order` test stays in place as an executable form of the design intent. * error.rs: replace the hand-rolled multi-variant enum and its manual Display / Error impls with a single `#[ohno::error]` struct that embeds OhnoCore. Construction goes through `CoverageGateError::new` for new errors and `::caused_by` for chained sources. Tests assert on the rendered message (which now includes the source chain via ohno's `Caused by:` format) rather than pattern-matching variants. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Two CI failures from the ohno refactor: * rustdoc treats `CoverageGateError::new` / `::caused_by` (generated by `#[ohno::error]`) as private items, so intra-doc links to them fail under `--D warnings`. Replace the broken links with plain backtick names in the doc comment. * `#[ohno::error]` implements `ohno::Enrichable` and `ohno::ErrorExt` on the error type, which transitively shows up in the public API. Add both traits to `[package.metadata.cargo_check_external_types] allowed_external_types` so the external-type-exposure check passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot wasn't able to review any files in this pull request.

Per reviewer feedback on PR #32: u32::MAX is ~4.3B lines, comfortably beyond anything a realistic codebase will ever produce, and u32 -> f64 is lossless because 32 bits fits inside the f64 mantissa with room to spare. Switching from u64 lets us drop the cast_precision_loss `#[expect]` and use `f64::from(...)` instead of an `as` cast. Affects: * LineCounters.{count,covered} and LineTotals.{count,covered}. * The aggregate sum is u32 throughout; no widening happens until the final percentage calculation, which uses `f64::from`. * Test helpers and fixtures updated to use u32 literals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

cargo-llvm-cov 0.6.x emits version 3.0.1 on recent toolchains; older releases emitted 2.0.1. The fields this parser reads (filename and lines counters) are unchanged across both, so both are now accepted without a warning. Discovered via dogfood on this workspace. Adds v3.json fixture, renames v2_is_supported / v3_is_supported tests, bumps the unknown_version fixture to 99.0.0, and updates design doc 10.4. 67 tests pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 2 comments.

Two related fixes from PR review: * verdict::classify checked totals.percent() before the threshold, so a crate with min-lines = 0.0 and no attributed coverage data ended up as Status::NoData (exit 2) instead of being opted out. Add a zero-threshold short-circuit at the top: if the threshold is <= epsilon, return Status::Ok regardless of whether any data was attributed. * design doc 4 corollary stated each member gets its own threshold or no threshold at all (silent opt-out), which contradicted the three-layer resolution (per-crate -> workspace -> built-in 100.0) and the explicit min-lines = 0.0 opt-out. Rewrite the corollary to match the actual behavior. Three new tests pin the fix: zero-threshold + no-data is Ok, zero-threshold + full-coverage is Ok, and an end-to-end evaluate() with an opted-out beta crate that has no attributed data passes with Verdict::Pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

sandersaares

Reviewed the design with first breath. Core idea looks solid, left some comments asking for clarification/refinement of design. Need to take another breath to look at impl.

…nology, naming clarity) Six related fixes from PR review: * Switch from crate to package as the unit of gating in the design doc, lib docs, README, CLI help, output strings, and rendered tables. The tool attributes by workspace member (a package), and the directory we read metadata from is the package's Cargo.toml. Internal Rust identifiers also follow: ThresholdSource::Crate -> Package, label crate -> package, Member.min_lines -> min_lines_percent, Workspace.default_min_lines -> default_min_lines_percent, the constant DEFAULT_MIN_LINES -> DEFAULT_MIN_LINES_PERCENT. Path strings like crates/ (the literal directory) and crates.io stay. * Rename min-lines metadata key to min-lines-percent. lines suggests a count; percent makes it explicit. Affects [package.metadata.coverage-gate] and [workspace.metadata.coverage-gate], the error messages, and every fixture / test that sets it. * Rename --json flag to --llvm-cov-json. The old name read like --string string; the new one says what the content is for. * Rename --crates flag to --packages, consistent with the package terminology and the env var pattern (IMPACTED_PACKAGES). * Drop the ox prefix from the rendered table header. Heading is now just coverage-gate (text) and ### coverage-gate (Markdown). * Reword section 5.4 to spell out what deterministic means - byte-identical input produces byte-identical output, not an API contract. * Reword section 6.2 to spell out that the unattributed-files warning is aggregated (one line per run with a count) so reviewers do not assume constant per-file noise. * Section 10.4 grows a Tooling requirements subsection telling adopters to run nightly Rust with cargo-llvm-cov >= 0.7 (older versions silently drop coverage(off) and inflate denominators). 70 tests pass, clippy clean. README regenerated via doc2readme. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 5 comments.

…surface Five follow-ups from PR review on the previous rename: * lib.rs binary-usage block listed --json and --crates (stale); now --llvm-cov-json and --packages, with min-lines-percent in the exit-code paragraph. * verdict.rs error message and matching test asserted on --crates; both now --packages so the mistyped-package diagnostic actually points users at a real flag. * threshold.rs intra-doc links referenced Member::min_lines and Workspace::DEFAULT_MIN_LINES_PERCENT; both were broken after the rename. Updated to Member::min_lines_percent and Workspace::default_min_lines_percent. * README referenced ./logo.png and ./favicon.ico from the shared README.j2 template but neither asset existed in this crate. Copied both from cargo-heather so the README renders. * Regenerated README via doc2readme so the auto-generated content matches the corrected lib.rs docs. 69 tests pass, clippy clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The previous rename batch missed a few spots. This pass sweeps everything else: * docs/implementation-plans/0000.md: stale references to min-lines, ### ox coverage-gate header, --crates flag, all updated to the new names. * lib.rs / verdict.rs: pub fn evaluate parameter renamed from gated_crates to gated_packages, plus doc comment references and the matching private resolve_gated() argument. Library callers see the new name immediately. * lib.rs / threshold.rs / workspace.rs doc comments: 'in the crates Cargo.toml' -> 'in the packages Cargo.toml', min-lines -> min-lines-percent, broken intra-doc link Threshold::min_lines -> Threshold::min_lines_percent. * tests/cli.rs: workspace_min_lines helper param renamed to workspace_min_lines_percent for consistency with the metadata key. * README regenerated via doc2readme so the public surface matches. Final grep for stale rename artifacts (--crates, ox coverage-gate, gated_crates, workspace_min_lines, Threshold::min_lines, DEFAULT_MIN_LINES) returns no matches. 70 tests pass, clippy clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 33 out of 34 changed files in this pull request and generated 2 comments.

…lcov Replace the cargo-llvm-cov JSON v2/v3 parser with the LCOV tracefile parser. This aligns the gate's line-coverage numbers with what codecov.io, ADO (via cobertura), and `cargo llvm-cov report --codecov` all report: lcov uses any-region-hit semantics on each line, while the JSON export uses the stricter every-region-hit rule and systematically under-reports coverage by 0.5-15pp. Changes: - Add `lcov` workspace dependency. - Replace `src/llvm_cov.rs` with `src/lcov_cov.rs` wrapping `lcov::Report`. - Rename `--llvm-cov-json` flag to `--lcov`; default path `target/coverage/coverage.json` -> `target/coverage/lcov.info`. - Replace JSON fixtures under `tests/fixtures/llvm_cov/` with lcov fixtures under `tests/fixtures/lcov/`. - Update design doc and implementation plan to describe lcov input and document the why-not-JSON rationale (section 6.1). - Regenerate README.md from updated lib.rs docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

martin-kolinek · 2026-05-28T13:04:14Z

🤖 Heads up: switched the gate's input format from cargo-llvm-cov JSON to LCOV tracefiles in 072bf18.

Why: while dogfooding on oxidizer-github we found that the JSON export systematically reports lower line coverage than codecov/ADO for the same source. Root cause: the JSON summary.lines uses strict every-region-hit semantics, whereas codecov, ADO (via cobertura), and cargo llvm-cov report --codecov all derive from LCOV which uses lenient any-region-hit. Concretely on the oxidizer-github workspace: JSON says 98.45%, LCOV says 99.26%, with per-file gaps of 5-15pp on macro-heavy files. Switching to LCOV makes the gate's numbers byte-comparable with what adopters see in their existing coverage UIs.

Flag rename: --llvm-cov-json -> --lcov; default path target/coverage/coverage.json -> target/coverage/lcov.info. Producer command becomes cargo llvm-cov report --lcov --output-path target/coverage/lcov.info.

The full rationale is now in design § 6.1.

_{AI response generated by Road to Gas Town}

… globs Match cargo build's package-selector idiom so the same impacted-package list can flow into `cargo build`, `cargo test`, and the coverage gate without translation: - Add `-p` short form for the existing `--packages` flag (now spelled `--package`); keep `--packages` as a hidden alias. - Allow the flag to be repeated, cargo-style: `-p foo -p bar`. The comma-separated form `-p foo,bar` still works. - Support Unix shell glob selectors: `-p 'cachet*'`, `-p '*macros'`, `-p 'a?pha'`. A selector that matches no workspace member is a configuration error (exit 2), mirroring cargo's behaviour. - Overlapping selectors de-duplicate, so `-p alpha -p 'alpha*'` produces a single row per matched member. Updates design § 5.2, implementation plan, lib.rs synopsis, and adds both unit tests for the glob matcher and integration tests for the repeated short form and glob selectors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 28 out of 30 changed files in this pull request and generated 3 comments.

+- Exit codes `0` / `1` / `2` per the design.
+- Loud-failure on zero data for a gated crate.
+
+Out of scope for this plan: function/region thresholds, lcov/cobertura


+
+The tool reads `Cargo.toml` files and a coverage lcov tracefile. It never
+writes; the only output channels are stdout and the optional summary
+file. No network access, no shell-out, no privileged operations.


+    #[arg(long = "lcov", value_name = "PATH")]
+    pub(crate) lcov: Option<PathBuf>,


Martin Kolinek (from Dev Box) and others added 16 commits May 25, 2026 20:00

Rename crate

1a37df2

docs(cargo-coverage-gate): rename design/0000.md to design/main.md

730371f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI added 3 commits May 26, 2026 20:48

martin-kolinek commented May 27, 2026

View reviewed changes

Comment thread crates/cargo-coverage-gate/src/aggregate.rs Outdated

martin-kolinek commented May 27, 2026

View reviewed changes

Comment thread crates/cargo-coverage-gate/src/error.rs Outdated

martin-kolinek marked this pull request as ready for review May 27, 2026 11:42

Copilot AI review requested due to automatic review settings May 27, 2026 11:42

Copilot started reviewing on behalf of martin-kolinek May 27, 2026 11:43 View session

Copilot AI reviewed May 27, 2026

View reviewed changes

Copilot stopped reviewing on behalf of martin-kolinek due to an error May 27, 2026 11:44
An unexpected error occurred. For more details, see the detailed logs in GitHub Actions.

Copilot AI review requested due to automatic review settings May 27, 2026 12:02

Copilot started reviewing on behalf of martin-kolinek May 27, 2026 12:02 View session

Copilot AI reviewed May 27, 2026

View reviewed changes

martin-kolinek enabled auto-merge (squash) May 27, 2026 12:55

kate-shine approved these changes May 27, 2026

View reviewed changes

Comment thread crates/cargo-coverage-gate/src/aggregate.rs Outdated

kate-shine approved these changes May 27, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings May 27, 2026 16:09

Copilot started reviewing on behalf of martin-kolinek May 27, 2026 16:09 View session

Copilot AI reviewed May 27, 2026

View reviewed changes

Comment thread crates/cargo-coverage-gate/src/verdict.rs Outdated

Comment thread crates/cargo-coverage-gate/docs/design/main.md Outdated

sandersaares reviewed May 28, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings May 28, 2026 11:31

Copilot started reviewing on behalf of martin-kolinek May 28, 2026 11:31 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

Copilot AI added 2 commits May 28, 2026 13:37

Copilot AI review requested due to automatic review settings May 28, 2026 11:48

Copilot started reviewing on behalf of martin-kolinek May 28, 2026 11:48 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

Comment thread crates/cargo-coverage-gate/src/verdict.rs Outdated

Comment thread crates/cargo-coverage-gate/docs/design/main.md Outdated

Copilot AI review requested due to automatic review settings May 28, 2026 15:35

Copilot started reviewing on behalf of martin-kolinek May 28, 2026 15:35 View session

Copilot AI reviewed May 28, 2026

View reviewed changes

		#[arg(long = "lcov", value_name = "PATH")]
		pub(crate) lcov: Option<PathBuf>,

Conversation

martin-kolinek commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

What it is not

Design highlights

Implementation

Test summary

Marked as draft

Uh oh!

github-actions Bot commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Breaking Changes Detected

Uh oh!

codecov-commenter commented May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

sandersaares left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

martin-kolinek commented May 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

martin-kolinek commented May 26, 2026 •

edited

Loading

github-actions Bot commented May 26, 2026 •

edited

Loading

codecov-commenter commented May 26, 2026 •

edited

Loading