-
Notifications
You must be signed in to change notification settings - Fork 0
feat(cargo-coverage-gate): add cargo subcommand for per-crate coverage gating #32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
martin-kolinek
wants to merge
30
commits into
main
Choose a base branch
from
cargo-coverage-gate
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
1d17275
Design doc for cargo-coverage-gate
1a37df2
Rename crate
f1e0d94
docs(cargo-coverage-gate): drop bump, add workspace default, default …
Copilot 2a11505
docs(cargo-coverage-gate): document cross-crate test attribution
Copilot 606b96f
docs(cargo-coverage-gate): restructure docs, add implementation plan …
Copilot 730371f
docs(cargo-coverage-gate): rename design/0000.md to design/main.md
Copilot 2d6efb3
docs(cargo-coverage-gate): drop the init subcommand
Copilot cb1b807
docs(cargo-coverage-gate): drop the check subcommand
Copilot c3c3501
feat(cargo-coverage-gate): phase 1 — crate skeleton
Copilot 15c45bd
feat(cargo-coverage-gate): phase 2 — JSON model
Copilot 0c9be5b
feat(cargo-coverage-gate): phase 3 — workspace discovery
Copilot dc5737c
feat(cargo-coverage-gate): phase 4 — threshold resolution
Copilot eb84cbb
feat(cargo-coverage-gate): phase 5 — attribution, aggregation, verdict
Copilot c595a0f
feat(cargo-coverage-gate): phase 6 — renderers
Copilot b70fb26
feat(cargo-coverage-gate): phase 7 — end-to-end CLI tests
Copilot c8fde9e
feat(cargo-coverage-gate): phase 8 — docs and release prep
Copilot 60f8c90
fix(cargo-coverage-gate): address PR check failures
Copilot 1c3b2fc
fix(cargo-coverage-gate): address remaining check failures
Copilot d192520
fix(cargo-coverage-gate): final spell-check fixes
Copilot d97ebff
fix(cargo-coverage-gate): spell-check tokenization edge cases
Copilot 84b0da2
refactor(cargo-coverage-gate): address PR review comments
Copilot 4746b60
fix(cargo-coverage-gate): rustdoc + external-type-exposure follow-ups
Copilot b8a3163
refactor(cargo-coverage-gate): switch line counters from u64 to u32
Copilot 084a4d5
feat(cargo-coverage-gate): accept LLVM coverage JSON v3 schema
Copilot dfae20e
fix(cargo-coverage-gate): honor min-lines = 0.0 as a true opt-out
Copilot b04cbee
refactor(cargo-coverage-gate): address PR review batch (package termi…
Copilot 063f828
fix(cargo-coverage-gate): align doc / error / asset with renamed CLI …
Copilot 496b7a3
docs(cargo-coverage-gate): finish the package terminology pass
Copilot 072bf18
feat(cargo-coverage-gate): switch input format from llvm-cov JSON to …
bd5dc9d
feat(cargo-coverage-gate): accept cargo-style `-p` / `--package` with…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| # Changelog |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
|
|
||
| [package] | ||
| name = "cargo-coverage-gate" | ||
| description = "A cargo subcommand that gates pull requests on per-package line coverage measured by cargo-llvm-cov" | ||
| version = "0.1.0" | ||
| readme = "README.md" | ||
| keywords = ["oxidizer", "cargo", "subcommand", "coverage", "ci"] | ||
| categories = ["command-line-utilities", "development-tools::cargo-plugins"] | ||
|
|
||
| edition.workspace = true | ||
| rust-version.workspace = true | ||
| authors.workspace = true | ||
| license.workspace = true | ||
| homepage.workspace = true | ||
| repository = "https://github.com/microsoft/ox-tools/tree/main/crates/cargo-coverage-gate" | ||
|
|
||
| [package.metadata.docs.rs] | ||
| all-features = true | ||
|
|
||
| [package.metadata.cargo_check_external_types] | ||
| allowed_external_types = [ | ||
| # The `#[ohno::error]` attribute on `CoverageGateError` implements | ||
| # these traits, which transitively appear in the public API. | ||
| "ohno::enrichable::Enrichable", | ||
| "ohno::error_ext::ErrorExt", | ||
| ] | ||
|
|
||
| [[bin]] | ||
| name = "cargo-coverage-gate" | ||
| path = "src/bin/cargo-coverage-gate/main.rs" | ||
|
|
||
| [dependencies] | ||
| cargo_metadata = { workspace = true } | ||
| clap = { workspace = true, features = ["derive", "std", "help", "usage", "error-context"] } | ||
| lcov = { workspace = true } | ||
| ohno = { workspace = true, features = ["app-err"] } | ||
| serde = { workspace = true, features = ["derive", "alloc"] } | ||
| serde_json = { workspace = true, features = ["std"] } | ||
| tracing = { workspace = true } | ||
| tracing-subscriber = { workspace = true, features = ["fmt"] } | ||
|
|
||
| [dev-dependencies] | ||
| assert_cmd = { workspace = true } | ||
| predicates = { workspace = true } | ||
| pretty_assertions = { workspace = true, features = ["std"] } | ||
| tempfile = { workspace = true } | ||
|
|
||
| [lints] | ||
| workspace = true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| <div align="center"> | ||
| <img src="./logo.png" alt="Cargo-Coverage-Gate Logo" width="96"> | ||
|
|
||
| # Cargo-Coverage-Gate | ||
|
|
||
| [](https://crates.io/crates/cargo-coverage-gate) | ||
| [](https://docs.rs/cargo-coverage-gate) | ||
| [](https://crates.io/crates/cargo-coverage-gate) | ||
| [](https://github.com/microsoft/ox-tools/actions/workflows/main.yml) | ||
| [](https://codecov.io/gh/microsoft/ox-tools) | ||
| [](../../LICENSE) | ||
| <a href="../.."><img src="../../logo.svg" alt="This crate was developed as part of the Oxidizer project" width="20"></a> | ||
|
|
||
| </div> | ||
|
|
||
| ## cargo-coverage-gate | ||
|
|
||
| A pull-request-time gate that compares per-package line coverage produced | ||
| by [`cargo-llvm-cov`][__link0] against per-package thresholds carried in | ||
| `Cargo.toml`. The accompanying `cargo-coverage-gate` binary reads the | ||
| coverage lcov tracefile, resolves each package’s threshold from a small | ||
| three-layer lookup, and emits a verdict table to stdout (and, | ||
| optionally, to a Markdown summary file for CI step summaries). | ||
|
|
||
| The full design is in [`docs/design/main.md`][__link1] in the source tree; | ||
| the implementation plan tracking the build is in | ||
| [`docs/implementation-plans/0000.md`][__link2]. | ||
|
|
||
| ### Threshold resolution | ||
|
|
||
| For each workspace member, the effective threshold is the first match | ||
| among: | ||
|
|
||
| 1. `[package.metadata.coverage-gate] min-lines-percent = N` in the package’s | ||
| `Cargo.toml`, | ||
| 1. `[workspace.metadata.coverage-gate] min-lines-percent = N` in the workspace | ||
| root `Cargo.toml`, or | ||
| 1. The built-in default of `100.0` — full coverage required. | ||
|
|
||
| Setting `min-lines-percent = 0.0` explicitly opts a package out of gating. | ||
|
|
||
| ### Why lcov, not the JSON? | ||
|
|
||
| `cargo-llvm-cov` exports the same instrumentation run in several | ||
| formats (JSON, lcov, cobertura, codecov-custom-JSON). The gate | ||
| consumes lcov because that is what every other coverage UI fed by | ||
| the same data sees: Codecov ingests lcov uploads directly, ADO | ||
| consumes cobertura that cargo-llvm-cov derives from lcov, and the | ||
| lcov line semantics (“a line is covered if any region on it was | ||
| hit”) match the human reading of “did we hit this line”. The JSON | ||
| export uses a stricter “every region on the line must be hit” | ||
| interpretation that systematically reports a couple of | ||
| percentage-points lower, which makes calibrating thresholds against | ||
| codecov / ADO numbers confusing. | ||
|
|
||
| ### Binary usage | ||
|
|
||
| ```text | ||
| cargo coverage-gate [--lcov <path>] [-p <spec>]... [--package <spec>]... | ||
| [--summary-file <path>] [--quiet] | ||
|
martin-kolinek marked this conversation as resolved.
|
||
| ``` | ||
|
|
||
| Exit codes: `0` if every gated package meets its threshold, `1` if any | ||
| gated package falls below its threshold, and `2` for configuration | ||
| errors (unparseable lcov, missing data for a gated package, a `--package` | ||
| selector that matches no member, an out-of-range `min-lines-percent` | ||
| value, …). | ||
|
|
||
| When `--summary-file` is unset, the binary falls back to | ||
| `$GITHUB_STEP_SUMMARY` and then `$COVERAGE_GATE_SUMMARY` to decide | ||
| where to write the Markdown verdict table. | ||
|
|
||
| ### Library usage | ||
|
|
||
| ```rust | ||
| use std::io; | ||
|
|
||
| let lcov = std::fs::read_to_string("target/coverage/lcov.info")?; | ||
| let report = cargo_coverage_gate::evaluate(&lcov, None, &[])?; | ||
| report.render_text(&mut io::stdout())?; | ||
| let code = report.verdict().exit_code(); | ||
| ``` | ||
|
|
||
| ### Public API | ||
|
|
||
| The library exposes [`evaluate`][__link3], which returns an | ||
| [`EvaluatedReport`][__link4]. The report can be rendered as plain text via | ||
| [`EvaluatedReport::render_text`][__link5] or as GitHub-flavored Markdown | ||
| via [`EvaluatedReport::render_markdown`][__link6], and reduced to a single | ||
| [`Verdict`][__link7] via [`EvaluatedReport::verdict`][__link8]. The accompanying | ||
| binary loads the lcov tracefile from disk and orchestrates rendering | ||
| plus the appropriate exit code. | ||
|
|
||
|
|
||
| <hr/> | ||
| <sub> | ||
| This crate was developed as part of <a href="../..">The Oxidizer Project</a>. Browse this crate's <a href="https://github.com/microsoft/ox-tools/tree/main/crates/cargo-coverage-gate">source code</a>. | ||
| </sub> | ||
|
|
||
| [__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjJhdIQbYLuo4OFUWT8bvMCT2d1BCU8bCvLHCBSvMr0bKR38GpAvnJ5hYvRhcoQbeMd-OXFcBm0bmRUrCNlaa-Mbh5mmtfAjdb0bXEoRTJRFNyZhZIGDc2NhcmdvLWNvdmVyYWdlLWdhdGVlMC4xLjBzY2FyZ29fY292ZXJhZ2VfZ2F0ZQ | ||
| [__link0]: https://github.com/taiki-e/cargo-llvm-cov | ||
| [__link1]: https://github.com/microsoft/ox-tools/blob/main/crates/cargo-coverage-gate/docs/design/main.md | ||
| [__link2]: https://github.com/microsoft/ox-tools/blob/main/crates/cargo-coverage-gate/docs/implementation-plans/0000.md | ||
| [__link3]: https://docs.rs/cargo-coverage-gate/0.1.0/cargo_coverage_gate/fn.evaluate.html | ||
| [__link4]: https://docs.rs/cargo-coverage-gate/0.1.0/cargo_coverage_gate/struct.EvaluatedReport.html | ||
| [__link5]: https://docs.rs/cargo-coverage-gate/0.1.0/cargo_coverage_gate/?search=EvaluatedReport::render_text | ||
| [__link6]: https://docs.rs/cargo-coverage-gate/0.1.0/cargo_coverage_gate/?search=EvaluatedReport::render_markdown | ||
| [__link7]: https://docs.rs/cargo-coverage-gate/0.1.0/cargo_coverage_gate/enum.Verdict.html | ||
| [__link8]: https://docs.rs/cargo-coverage-gate/0.1.0/cargo_coverage_gate/?search=EvaluatedReport::verdict | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.