Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ cargo-gamma-process = { path = "crates/cargo-gamma-process", default-features =
cargo-gamma-rt = { path = "crates/cargo-gamma-rt", default-features = false, version = "0.1.0" }
cargo-gamma-unsafe = { path = "crates/cargo-gamma-unsafe", default-features = false, version = "0.1.0" }
cargo-heather = { path = "crates/cargo-heather", default-features = false, version = "0.2.1" }
# external dependencies
cargo-platform = { version = "0.3.3", default-features = false }
cargo_metadata = { version = "0.23.1", default-features = false }
cel = { version = "0.14.4", default-features = false }
chrono = { version = "0.4.45", default-features = false }
Expand Down
23 changes: 23 additions & 0 deletions crates/cargo-coverage-gate/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1 +1,24 @@
# Changelog
## [0.4.0] - 2026-08-28

- ✨ Features

- add target-specific policies
- add cargo-aprz and cargo-ensure-no-default-features ([#76](https://github.com/microsoft/ox-tools/pull/76))
- run a command per workspace member with cargo-style selection ([#61](https://github.com/microsoft/ox-tools/pull/61))
- add expect-no-coverable-lines assertion ([#51](https://github.com/microsoft/ox-tools/pull/51))
Comment thread
martin-kolinek marked this conversation as resolved.

- 🐛 Bug Fixes

- preserve cross-package coverage
- resolve effective target policies lazily

- 📚 Documentation

- document configuration capabilities

- ♻️ Code Refactoring

- tighten target policy contracts
- reuse thresholds for target opt-outs

3 changes: 2 additions & 1 deletion crates/cargo-coverage-gate/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[package]
name = "cargo-coverage-gate"
description = "A cargo subcommand that gates pull requests on per-package line coverage measured by cargo-llvm-cov"
version = "0.3.0"
version = "0.4.0"
readme = "README.md"
Comment thread
martin-kolinek marked this conversation as resolved.
keywords = ["oxidizer", "cargo", "subcommand", "coverage", "ci"]
categories = ["command-line-utilities", "development-tools::cargo-plugins"]
Expand All @@ -31,6 +31,7 @@ name = "cargo-coverage-gate"
path = "src/bin/cargo-coverage-gate/main.rs"

[dependencies]
cargo-platform = { workspace = true }
cargo_metadata = { workspace = true }
clap = { workspace = true, features = ["derive", "std", "help", "usage", "error-context"] }
lcov = { workspace = true }
Expand Down
128 changes: 103 additions & 25 deletions crates/cargo-coverage-gate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,9 @@
A pull-request-time gate that compares per-package line coverage produced
by [`cargo-llvm-cov`][__link0] against per-package thresholds carried in
`Cargo.toml`. The accompanying `cargo-coverage-gate` binary reads the
coverage lcov tracefile, resolves each package’s threshold from a small
three-layer lookup, and emits a verdict table to stdout (and,
coverage lcov tracefile, resolves each package’s base policy from a small
three-layer lookup, applies any matching package target policy, and emits a
verdict table to stdout (and,
optionally, to a Markdown summary file for CI step summaries). A failing
verdict includes actionable details without relying on a later
coverage-service upload. A coverable line is a distinct LCOV `DA:` record.
Expand All @@ -28,9 +29,27 @@ Numeric failures show exact covered/coverable counts and uncovered ranges;
and `NO DATA` explains that no records were attributed. Location lists are
bounded, with an exact count of omitted locations.

### Threshold resolution
### Configuration

For each workspace member, the effective threshold is the first match
#### Numeric thresholds

A workspace can define the default line-coverage threshold:

```toml
# Illustrative workspace policy.
[workspace.metadata.coverage-gate]
min-lines-percent = 80
```

Individual packages can override it:

```toml
# Illustrative package policy, intentionally stricter than the workspace.
[package.metadata.coverage-gate]
min-lines-percent = 95
Comment thread
martin-kolinek marked this conversation as resolved.
```

For each workspace member, the base threshold is the first match
among:

1. `[package.metadata.coverage-gate] min-lines-percent = N` in the package’s
Expand All @@ -40,13 +59,67 @@ among:
1. The built-in default of `100.0` — full coverage required.

Setting `min-lines-percent = 0.0` explicitly opts a package out of
gating (it always passes, regardless of attributed data). A package
that legitimately contains no coverable lines (pure re-exports, type
definitions, a thin binary shim) instead declares
`expect-no-coverable-lines = true`: the gate passes only while that
holds and fails — as a regression — if coverable lines later appear.
The two keys are mutually exclusive, and `expect-no-coverable-lines`
is package-scoped only.
gating: it always passes, regardless of attributed data. Thresholds must
be in the inclusive range `0.0..=100.0`.

#### Packages with no coverable lines

A package that legitimately contains no coverable lines (pure re-exports,
type definitions, or a thin binary shim) can make that invariant explicit:

```toml
[package.metadata.coverage-gate]
expect-no-coverable-lines = true
```

The gate passes only while the package has no attributed coverable lines
and fails as a regression if coverable code later appears. This differs
from `min-lines-percent = 0`, which keeps passing if the package grows
coverable code. The two keys are mutually exclusive, and
`expect-no-coverable-lines` is package-scoped only.

#### Target-specific policies

A package can replace its base policy for a Cargo-style target selector:

```toml
[package.metadata.coverage-gate]
min-lines-percent = 100

[package.metadata.coverage-gate.target.'cfg(not(windows))']
expect-no-coverable-lines = true

[package.metadata.coverage-gate.target.x86_64-unknown-linux-gnu]
min-lines-percent = 100
```

A target-specific no-coverable-lines assertion uses the same nesting:

```toml
[package.metadata.coverage-gate.target.thumbv7em-none-eabihf]
expect-no-coverable-lines = true
```

Target tables are package-scoped; they are invalid in workspace metadata.
Their keys accept exact Rust target triples or quoted `cfg(...)` expressions
using the target-derived subset of Cargo’s target grammar. Target
configuration options such as `windows`, `unix`, `target_os`, and
`target_arch` are supported. Build-context options such as `feature`, `test`,
`debug_assertions`, and `proc_macro` are rejected because a standalone target
query cannot evaluate them. A selected target table sets either
`min-lines-percent` or `expect-no-coverable-lines = true`, completely
replacing the package’s base policy to produce its effective policy. Exact
triples take precedence over matching `cfg(...)` expressions. Multiple
matching cfg policies are a configuration error rather than depending on
declaration order.

A zero target-specific threshold disables gating on the matching target,
but does not disable test execution or instrumentation. Those test binaries
remain instrumented because they may contribute coverage to other packages.
If cargo-llvm-cov reports that an instrumented run produced no coverage
data, automation can supply an empty lcov tracefile: zero-threshold and
`expect-no-coverable-lines` packages pass, while positively gated packages
report `NO DATA`.

### Why lcov, not the JSON?

Expand All @@ -66,6 +139,7 @@ Codecov / ADO numbers confusing.

```text
cargo coverage-gate [--lcov <path>]... [-p|--package <spec>]...
[--target <triple>]
[--summary-file <path>] [--quiet]
```

Expand Down Expand Up @@ -97,25 +171,29 @@ let code = report.verdict().as_exit_code();

### Public API

The library exposes [`evaluate`][__link1], which returns an
[`EvaluatedReport`][__link2]. The report can be rendered as plain text via
[`EvaluatedReport::render_text`][__link3] or as GitHub-flavored Markdown
via [`EvaluatedReport::render_markdown`][__link4], and reduced to a single
[`Verdict`][__link5] via [`EvaluatedReport::verdict`][__link6]. The accompanying
binary loads the lcov tracefile from disk and orchestrates rendering
plus the appropriate exit code.
[`evaluate`][__link1] gates one lcov tracefile for the rustc host target, while
[`evaluate_many`][__link2] merges multiple tracefiles at line level.
[`evaluate_many_for_target`][__link3] evaluates a selected Rust target, which may be
supplied explicitly or omitted to select the rustc host target.
Evaluation returns an [`EvaluatedReport`][__link4], which renders as plain
text via [`EvaluatedReport::render_text`][__link5] or GitHub-flavored Markdown via
[`EvaluatedReport::render_markdown`][__link6] and reduces to a [`Verdict`][__link7] via
[`EvaluatedReport::verdict`][__link8]. The accompanying binary loads tracefiles from
disk and orchestrates rendering plus the appropriate exit code.


<hr/>
<sub>
This crate was developed as part of <a href="../..">The Oxidizer Project</a>. Browse this crate's <a href="https://github.com/microsoft/ox-tools/tree/main/crates/cargo-coverage-gate">source code</a>.
</sub>

[__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjNhdIQbFhzZ8rzWNNYbuRaDSGWynFgbH4PMdoT7GNcbVwNPtPjAhvFhYvRhcoQbSLvuLqYvzncbvkJE0ZCfihkbvLj44R1A234bYHxJKfzepgZhZIGDc2NhcmdvLWNvdmVyYWdlLWdhdGVlMC4zLjBzY2FyZ29fY292ZXJhZ2VfZ2F0ZQ
[__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjNhdIQbFhzZ8rzWNNYbuRaDSGWynFgbH4PMdoT7GNcbVwNPtPjAhvFhYvRhcoQb3wjNoVxGaCAbpkmpjr98NCcbw-HRsqJQXfkb8-afvWiSredhZIGDc2NhcmdvLWNvdmVyYWdlLWdhdGVlMC40LjBzY2FyZ29fY292ZXJhZ2VfZ2F0ZQ
[__link0]: https://github.com/taiki-e/cargo-llvm-cov
[__link1]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/fn.evaluate.html
[__link2]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/struct.EvaluatedReport.html
[__link3]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/?search=EvaluatedReport::render_text
[__link4]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/?search=EvaluatedReport::render_markdown
[__link5]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/enum.Verdict.html
[__link6]: https://docs.rs/cargo-coverage-gate/0.3.0/cargo_coverage_gate/?search=EvaluatedReport::verdict
[__link1]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/fn.evaluate.html
[__link2]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/fn.evaluate_many.html
[__link3]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/fn.evaluate_many_for_target.html
[__link4]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/struct.EvaluatedReport.html
[__link5]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/?search=EvaluatedReport::render_text
[__link6]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/?search=EvaluatedReport::render_markdown
[__link7]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/enum.Verdict.html
[__link8]: https://docs.rs/cargo-coverage-gate/0.4.0/cargo_coverage_gate/?search=EvaluatedReport::verdict
Loading
Loading