chore: cargo-ox-check, unified Rust build/CI scaffolding tool by martin-kolinek · Pull Request #25 · microsoft/ox-tools

martin-kolinek · 2026-05-13T18:46:04Z

Design document for cargo-ox-ci, an opinionated Rust build/CI scaffolding tool intended to unify build infrastructure across the Oxidizer family of repos (oxidizer, oxidizer-github, ox-tools, ox-tools-gh, assistants-oxide, ox-docs) and ship as OSS via crates.io.

No code yet — this PR introduces a binary-crate skeleton (crates/cargo_ox_ci/) and the design under crates/cargo_ox_ci/docs/design/. cargo check -p cargo-ox-ci passes; the binary currently just exits with a "not yet implemented" message.

Design summary

Six documents under crates/cargo_ox_ci/docs/design/:

design.md — problem, goals, non-goals, guiding principle, UX, repo layout, cross-cutting concerns (security, cross-OS matrices, internal-vs-OSS).
checks.md — opinionated check catalog: 3 PR groups (pr-fast, pr-test, pr-mutants) + 4 nightly groups (nightly-test, nightly-advisories, nightly-runtime, nightly-exhaustive); per-check invocations; impact-scoping check → env-var mapping.
local.md — justfiles/ox-ci/ recipe tree, tool-version policy, pass-through env-var protocol (excludes + skip), customization at the recipe level.
updates.md — sidecar manifest (.ox-ci.lock), checksum-driven three-way state machine, dirty-file handling, opt-out via emptiness, proposal full-file emission.
github.md — three-layer emission model: root workflows (owned), reusable workflows ox-ci-{pr,nightly}-impl.yml (owned), per-group composite actions (owned). Uniform exclude+skip inputs.
ado.md — three-layer emission model: root pipelines (owned), stages templates ox-ci/{pr,nightly}.yml (owned), per-group step templates (owned). 1ESPT composition stays user-side; ox-ci's templates contain no compliance-harness references.

Key design choices

just as the local runtime; CI invokes the same recipes. No tool dependency at CI runtime.
cargo-delta impact scoping passed via env vars; recipes choose what to consume from the catalog.
Cross-OS matrix (Linux + Windows by default) for the test-running groups; everything else Linux-only. Configurable per-repo via one input.
Sidecar .ox-ci.lock tracks last-rendered checksums per owned file and per managed region. Eliminates in-file checksum noise and makes claim-with-no-upstream-change a silent no-op.
Proposals are emitted as full ready-to-use files (<path>.ox-ci-proposed) even for region-scoped updates, so users can diff/mv cleanly.
pwsh is the single non-cargo runtime dep (used by pr-title only; other previously-scripted checks license-headers/ensure-no-* are plain cargo subcommands from the ox-tools family).
TOML managed regions use a single parent header + dotted-key body to accommodate TOML's no-duplicate-tables rule, so users can extend lints/etc. from below the closing sentinel.

Status

Draft. Open for feedback on:

Crate name (cargo-ox-ci) — confirm before crates.io publication.
Catalog location (currently hardcoded; externalizable later).
pr-title regex (Conventional Commits, hardcoded with disable escape).
Whether to add a future cargo ox-ci-pr-title cargo subcommand to drop the pwsh dep entirely.

Codecov Comments Bot (codecov-commenter) · 2026-05-13T18:47:59Z

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.4%. Comparing base (ea917bf) to head (2ed27b1).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/cargo_ox_check/src/main.rs	0.0%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##            main     #25      +/-   ##
========================================
- Coverage   99.6%   86.4%   -13.2%     
========================================
  Files         11      16       +5     
  Lines       1847     665    -1182     
========================================
- Hits        1840     575    -1265     
- Misses         7      90      +83

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The previous text claimed ust-toolchain.toml-driven auto-install was cached across subsequent runs. It is not — ~/.rustup is not in the cache key set by ox-ci-setup, so the toolchain download runs once per job. Cost is small relative to the cached cargo registry / target paths, so the design choice is fine, but the docs shouldn't claim a cache hit that doesn't exist. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Microsoft/ox-tools (where cargo-ox-ci lives) uses `cargo ox-ci update` to manage its own CI. Every PR runs the regenerate-check gate: build the binary from source, run `update`, `git diff --exit-code`. A catalog or emitter change that produces drift fails its own PR check immediately. Layers documented: - Self-hosting (primary): the dogfood loop. - Fixture-based integration tests for shapes ox-tools doesn't have. - Schema validation (actionlint / taplo / just --summary). - Manual release checklist for things dogfooding can't catch (1ESPT, self-hosted runners, macOS, cross-repo migrations). Also covers bootstrap (the one hand-written workflow that wraps the regenerated reusable workflow), breaking-change handling, recovery from self-inflicted main breakage, and acknowledged coverage gaps. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

A fix PR builds cargo-ox-ci from its own branch, so its regenerate-check passes regardless of main's state. What's actually blocked is unrelated in-flight PRs whose merge base is the broken commit; they recover by rebasing past the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Sander Saares (sandersaares)

I like it and want to try it out already.

Sander Saares (sandersaares) · 2026-05-14T06:55:11Z

+
+1. **One opinionated build profile** for Rust repos, with sane defaults distilled from the
+   strongest patterns observed across the existing repos.
+2. **Two tiers**: `pr` (blocking on every pull request) and `nightly` (slow, scheduled).


I would call these validate-mini, validate and validate-full to emphasize what they do rather than what triggers them (which might ultimately be customized by users).

My thinking was exactly to tie them to the triggers - one of the common complaints we see is that there isn't sufficient clarity on what to run to get a green PR build. I'd prefer to stick the "pr/nightly" into the recipe name to clearly make this connection.

My feeling is that the gap is "there is no command" and not that "the name does not say is-pr-green" so the tradeoff seems dubious. Well, at least we should call it "scheduled" and not "nightly" - that might help as well.

Will rename nightly to scheduled, that one I buy. The question of which command to run is important in my opinion. I can live with ox-check (for the PR) and ox-check-scheduled (for the scheduled/nightly).

Addresses review feedback (#25) about `CI` misuse for what is really a local+CI validation tool rather than continuous-integration methodology. The rename is identifier-scoped; tier names (`pr`, `nightly`, `full`) and prose use of `CI` stay since the trade-off there is less clear-cut. Renamed: - crate: cargo-ox-ci -> cargo-ox-check (binary, package, folder) - recipes: ox-ci-* -> ox-check-* (ox-check-pr, ox-check-clippy, ...) - region IDs: ox-ci-managed: ox-ci-* -> ox-check-managed: ox-check-* - env vars: OX_CI_* -> OX_CHECK_* - manifest: .ox-ci.lock -> .ox-check.lock - proposal files: .ox-ci-proposed -> .ox-check-proposed Not renamed: 'CI' in prose, 'ci' in crates.io keywords, the description string's industry-shorthand use of CI scaffolding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Addresses review feedback (#25): the previous `--backend github|ado|both|none` surface bakes in the assumption of exactly two backends. Replace with a repeatable `--backend <name>` flag (clap `Vec<Backend>`) plus a `--no-backends` switch for the CI-less case. Adding a third backend in the future is now a catalog change, not a CLI-syntax change. Semantics preserved: - omitted -> autodetect from origin URL (unchanged) - --backend github --backend ado -> both (was: --backend both) - --no-backends -> emit no CI files (was: --backend none) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Sander Saares (sandersaares)

Partial review pass - ran out of time today. Will continue at next opportunity.

Sander Saares (sandersaares) · 2026-06-02T12:21:11Z

+
+1. **One opinionated build profile** for Rust repos, with sane defaults distilled from the
+   strongest patterns observed across the existing repos.
+2. **Two tiers**: `pr` (blocking on every pull request) and `nightly` (slow, scheduled).


My feeling is that the gap is "there is no command" and not that "the name does not say is-pr-green" so the tradeoff seems dubious. Well, at least we should call it "scheduled" and not "nightly" - that might help as well.

Sander Saares (sandersaares) · 2026-06-02T12:25:48Z

+impact scoping in `oxidizer-github`) take months to propagate, security/policy upgrades are
+missed, and onboarding new Rust repos requires copying-and-praying.
+
+## 2. Goals


I would add "automated tooling setup in local PC and cloud pipelines" as a goal. If some setup is needed for ox-check workflows to work, it is ox-check that should be setting that up.

Sander Saares (sandersaares) · 2026-06-02T12:26:31Z

+impact scoping in `oxidizer-github`) take months to propagate, security/policy upgrades are
+missed, and onboarding new Rust repos requires copying-and-praying.
+
+## 2. Goals


We know that Cargo tool versions are as critical to reproducible validation results as Cargo toolchain versions. Do we intend to solve repo-local tool installs as part of the scope of this suite? Today we run the risk of someone working in 3 repos needing to (re)install tools for each repo when switching if they e.g. contain incompatible versions of cargo-mutants, cargo-doc2readme or similar correctness-critical tools.

In principle, cargo install can install to a repo-local path, so the problem reduces to solving the "running" side which might be doable with just some logic if we focus on justfiles as the commanding entrypoint (which we appear to do here).

This might also tie into point 5 in the list, as local-cloud reproducibility already assumes matching tools. We just need to guard against neighbors on the same local PC.

Do we intend to solve repo-local tool installs as part of the scope of this suite?

I would hope that wouldn't be necessary for two reasons:

We'd hope most users won't need to customize the tools they need

If they do, they can build on top of the just recipes provided by the tool

In principle, cargo install can install to a repo-local path, so the problem reduces to solving the "running" side which might be doable with just some logic if we focus on justfiles as the commanding entrypoint (which we appear to do here).

I won't do this in the first iteration to avoid some of the complexity. My hypothesis is that most tools get stricter with increasing versions, not less strict, so satisfying a newer version that the user has installed should cover the cloud check. That said, if this hypothesis proves to be false, we can add local installs and separate the ox-check tools from the tools installed by the user globally (although looking at the cargo install docs, it doesn't seem like it's fully supported and we'd need to do a bunch of customization).

Sander Saares (sandersaares) · 2026-06-02T12:30:50Z

+- Owning `.cargo/config.toml`, `rust-toolchain.toml`, or workspace layout in `Cargo.toml`.
+- Installing the Rust toolchain. msrustup owns it on 1ESPT; the runner image owns it on
+  GitHub-hosted runners; the user owns it locally. The tool validates `rustc` version at
+  recipe time and produces a clean failure when it doesn't meet the catalog minimum.


Different toolchain versions have different behavior, though - different Clippy rules, for example. This seems to contradict the idea of reproducible local-cloud results. I understand we cannot own the installation but we should not be satisfied with only the minimum version - I think we need to require an exact match (at least up to the minor version).

Some commands also require nightly toolchains to function. I suppose we can just let the user own the version and as long as ox-check logic enforces the version is always in the command string, Cargo will complain if it is missing?

For repos that specify rust-toolchain, this results in full reproducibility. For repos which don't, there isn't much we can do. I don't think we can enforce rust-toolchain in all repos (for example, we don't specify the toolchain in our github repos since the internal toolchain is different from the external one). So we accept some drift with the hope that issues will be rare enough. That said, I'll look into the ability to warn on difference between the toolchain installed by the CI setup and the local one.

Sander Saares (sandersaares) · 2026-06-02T12:32:39Z

+- Installing the Rust toolchain. msrustup owns it on 1ESPT; the runner image owns it on
+  GitHub-hosted runners; the user owns it locally. The tool validates `rustc` version at
+  recipe time and produces a clean failure when it doesn't meet the catalog minimum.
+- Managing exact tool versions on the user's behalf — we enforce minimums only. See


This feels like a big surrender to me. Why? Tool versions certainly impact validation results - without tool versioning, I feel we are undermining the premise of reproducible cloud-local behavior.

See my response above - I hope enforcing minimums should be enough to achieve one-way reproducibility - if things pass locally, they are very likely to pass in cloud.

Sander Saares (sandersaares) · 2026-06-02T13:07:02Z

+## 3. Managed regions
+
+Co-owned files with one or more tool-managed sections delimited by sentinel comments.
+Sentinels persist; the inner `# checksum:` line that earlier designs carried no longer


Confusing to mention historical designs in a design document, I suggest removing this statement.

Note: some examples in other .md files still contain inline checksums - should remove them.

Sander Saares (sandersaares) · 2026-06-02T13:08:58Z

+clippy.expect_used = "deny"
+rustdoc.broken_intra_doc_links = "deny"
+# <<< ox-check-managed: ox-check-workspace-lints
+# User-added lints continue [workspace.lints] via dotted keys — valid TOML, ox-check


I have a feeling we also need to include an ox-check region into AGENTS.md to ensure any agents are informed on how to deal with our special sections in such situations (but also perhaps in general to provide them guidance and hyperlinks to how to deal with ox-check).

Yes. AGENTS.md are out of scope in the first version, but we'll add it.

Sander Saares (sandersaares) · 2026-06-02T13:35:51Z

+```
+
+Whitespace and comments inside the region are preserved verbatim by the rewrite — the
+tool computes the checksum over the body bytes literally.


Are line ending characters considered part of the body bytes?

No, they get normalized before computing the checksum

Sander Saares (sandersaares) · 2026-06-02T13:43:20Z

+Whitespace and comments inside the region are preserved verbatim by the rewrite — the
+tool computes the checksum over the body bytes literally.
+
+### User-extension limits for TOML regions


Do we foresee common scenarios where editing the owned regions will be required by users?

Project which want to have a different set of lints enabled? I hope these will be rare.

Sander Saares (sandersaares) · 2026-06-02T13:53:15Z

+
+| Host                                  | Region                                  | Anchor for insertion                                          |
+|---------------------------------------|-----------------------------------------|---------------------------------------------------------------|
+| `Justfile`                            | `ox-check-imports`                         | After leading `set …` / shebang lines, before the first user recipe. |


What is the significance of this going to the top?

martin-kolinek · 2026-06-04T08:33:35Z

Closing this PR, continue any conversations in #33

PR #25 feedback (sandersaares): the CI tier name "nightly" conflates two distinct concepts -- Rust's nightly toolchain and "runs on a schedule" -- and confuses adopters. Rename the tier wholesale to "scheduled" and preserve all genuine references to Rust's nightly toolchain. Renamed: - Recipes: ox-check-nightly -> ox-check-scheduled, plus all four group recipes (nightly-{test,advisories,runtime,exhaustive} -> scheduled-*). - Template files: ado/nightly-stages.yml -> ado/scheduled-stages.yml, ado/nightly-root-pipeline.yml -> ado/scheduled-root-pipeline.yml, github/nightly-{impl,root}-workflow.yml -> github/scheduled-*. - Emitted file paths: .github/workflows/ox-check-nightly{,.impl}.yml -> ox-check-scheduled{,.impl}.yml, .pipelines/ox-check-nightly.yml -> ox-check-scheduled.yml, .pipelines/ox-check/nightly.yml -> ox-check/scheduled.yml. - Rust constants: NIGHTLY_STAGES -> SCHEDULED_STAGES, NIGHTLY_*_WORKFLOW -> SCHEDULED_*_WORKFLOW, NIGHTLY_ROOT_PIPELINE -> SCHEDULED_*. - ADO stage names: nightly_{test,advisories,runtime,exhaustive} -> scheduled_*. - Codecov flag on the upload step: `flags: nightly` -> `flags: scheduled` so streams stay distinguishable in the Codecov UI. - Docs: all CI-tier references in design.md, checks.md, ado.md, github.md, updates.md, README.md, implementation-plans/. Kept (genuine Rust-nightly-toolchain references): - versions.just `rust_nightly`, `rust_nightly_external_types` variables and their installation scripts in setup composites. - `cargo +nightly` / `cargo +nightly-2026-02-10` invocations in recipes. - "nightly rustfmt", "pinned nightly toolchains", "nightly rustdoc", "Nightly pinning" section in local.md, `coverage_nightly` cfg in lints body, etc. Also addressed in this commit (also PR #25 feedback): - design.md §2 ("Non-Goals"): add follow-up note about warning on local toolchain drift from the catalog-targeted version. - design.md §8.3 ("Cross-OS test matrices"): add caveat that cargo-delta does not currently model OS-conditional dependency graphs, and that the scheduled tier's full-workspace run is the safety net. - design.md §8.1 ("Security"): clarify the pr-fast reference so first-time readers don't have to chase forward to §8.3 / checks.md. - updates.md §3 ("Managed regions"): explicitly document line-ending normalization in checksum computation, and that whitespace-stripping editors will register as a user edit. - updates.md §3: drop the dangling "earlier designs carried a checksum line" sentence (vestigial).

Addresses review feedback (#25) about `CI` misuse for what is really a local+CI validation tool rather than continuous-integration methodology. The rename is identifier-scoped; tier names (`pr`, `nightly`, `full`) and prose use of `CI` stay since the trade-off there is less clear-cut. Renamed: - crate: cargo-ox-ci -> cargo-ox-check (binary, package, folder) - recipes: ox-ci-* -> ox-check-* (ox-check-pr, ox-check-clippy, ...) - region IDs: ox-ci-managed: ox-ci-* -> ox-check-managed: ox-check-* - env vars: OX_CI_* -> OX_CHECK_* - manifest: .ox-ci.lock -> .ox-check.lock - proposal files: .ox-ci-proposed -> .ox-check-proposed Not renamed: 'CI' in prose, 'ci' in crates.io keywords, the description string's industry-shorthand use of CI scaffolding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Addresses review feedback (#25): the previous `--backend github|ado|both|none` surface bakes in the assumption of exactly two backends. Replace with a repeatable `--backend <name>` flag (clap `Vec<Backend>`) plus a `--no-backends` switch for the CI-less case. Adding a third backend in the future is now a catalog change, not a CLI-syntax change. Semantics preserved: - omitted -> autodetect from origin URL (unchanged) - --backend github --backend ado -> both (was: --backend both) - --no-backends -> emit no CI files (was: --backend none) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

PR #25 feedback (sandersaares): the CI tier name "nightly" conflates two distinct concepts -- Rust's nightly toolchain and "runs on a schedule" -- and confuses adopters. Rename the tier wholesale to "scheduled" and preserve all genuine references to Rust's nightly toolchain. Renamed: - Recipes: ox-check-nightly -> ox-check-scheduled, plus all four group recipes (nightly-{test,advisories,runtime,exhaustive} -> scheduled-*). - Template files: ado/nightly-stages.yml -> ado/scheduled-stages.yml, ado/nightly-root-pipeline.yml -> ado/scheduled-root-pipeline.yml, github/nightly-{impl,root}-workflow.yml -> github/scheduled-*. - Emitted file paths: .github/workflows/ox-check-nightly{,.impl}.yml -> ox-check-scheduled{,.impl}.yml, .pipelines/ox-check-nightly.yml -> ox-check-scheduled.yml, .pipelines/ox-check/nightly.yml -> ox-check/scheduled.yml. - Rust constants: NIGHTLY_STAGES -> SCHEDULED_STAGES, NIGHTLY_*_WORKFLOW -> SCHEDULED_*_WORKFLOW, NIGHTLY_ROOT_PIPELINE -> SCHEDULED_*. - ADO stage names: nightly_{test,advisories,runtime,exhaustive} -> scheduled_*. - Codecov flag on the upload step: `flags: nightly` -> `flags: scheduled` so streams stay distinguishable in the Codecov UI. - Docs: all CI-tier references in design.md, checks.md, ado.md, github.md, updates.md, README.md, implementation-plans/. Kept (genuine Rust-nightly-toolchain references): - versions.just `rust_nightly`, `rust_nightly_external_types` variables and their installation scripts in setup composites. - `cargo +nightly` / `cargo +nightly-2026-02-10` invocations in recipes. - "nightly rustfmt", "pinned nightly toolchains", "nightly rustdoc", "Nightly pinning" section in local.md, `coverage_nightly` cfg in lints body, etc. Also addressed in this commit (also PR #25 feedback): - design.md §2 ("Non-Goals"): add follow-up note about warning on local toolchain drift from the catalog-targeted version. - design.md §8.3 ("Cross-OS test matrices"): add caveat that cargo-delta does not currently model OS-conditional dependency graphs, and that the scheduled tier's full-workspace run is the safety net. - design.md §8.1 ("Security"): clarify the pr-fast reference so first-time readers don't have to chase forward to §8.3 / checks.md. - updates.md §3 ("Managed regions"): explicitly document line-ending normalization in checksum computation, and that whitespace-stripping editors will register as a user edit. - updates.md §3: drop the dangling "earlier designs carried a checksum line" sentence (vestigial).

- Strip pre-existing spellcheck.toml content that collides with managed region - Fix stale snapshots + spellcheck typos from recent commits - ADO impact.yml needs setup.yml to bootstrap cargo - Cross-distro libclang install in setup.yml - Env-var form for impact baseline ref selection - Skip LFS smudge in impact baseline worktree - Validate impact package names; drop unknowns - Manifest mutation tests (double-newline guard, loop-bound progress) - Unify semver-check workspace path with per-package tolerance - Add ox-check-system-deps-check: probe for system-level libs (libclang) with per-OS install hints - Rename "nightly" tier to "scheduled" (PR #25 review feedback) - Per-OS impact stages to fix cfg-conditional dep gaps (impact-linux + impact-windows; each downstream leg consumes its OS's impact set) - Local-vs-CI parity for readme-check + impact validation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Design and planning for cargo-ox-check-update Top-level design document covering the why (unified, opinionated CI/build scaffolding for Rust repos), principles (recipe-first; cargo-as-fallback; owned-with-customization), and tier/group structure. Companion documents split out checks.md (catalog), local.md (justfile layout), updates.md (drift detection), github.md / ado.md (per-backend wiring), and verification.md (continuous-validation strategy). Includes the initial pipeline design, the --backend flag shape (repeatable not enum), the rename from cargo-ox-ci to cargo-ox-check, and implementation-plans/0000.md laying out the 22-step MVP path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * MVP implementation: CLI, manifest, decision algorithm, GH/ADO emitters 22 implementation steps from plan 0000: 1. CLI surface with no-op `update` subcommand 2. Backend autodetection from `origin` URL 3. Workspace discovery and member enumeration 4. `.ox-check.lock` manifest read/write 5. SHA-256 helpers + three-checksum (T/L/D) decision algorithm 6. Managed-region parser and writer 7. Plan, proposed-file emission, dry-run summary 8. tools.just emitter and owned-file driver 9. checks.just emitter (per-check recipes) 10. groups.just + tiers.just emitters 11. Justfile imports managed region 12. Cargo.toml workspace + member [lints] regions 13. deny.toml, rustfmt.toml, .delta.toml regions 14. End-to-end local-only update wiring 15-17. GitHub composite actions, reusable workflows, root workflows 18-20. ADO step templates, stages templates, root pipelines 21. Schema-validation tests (actionlint, taplo, just-parse) 22. regenerate-check workflow for dogfooding Plus initial README and crates.io polish. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Post-MVP refactors and snapshot tests - Switch from anyhow to ohno::AppError throughout cargo-ox-check - Drop DecisionInputs::emptied and Decision::Skipped - Drop spurious ox-check-managed sentinels from owned .just templates - Move per-group GH action / ADO step YAML bodies into template files (single source for the per-group template body, substituted at emit) - Single-import Justfile pattern: only mod.just visible to users, all ox-check recipes reached transitively; alias in mod.just - Drop ADO YAML `pr:` trigger (Azure Repos uses branch policies) - Include master alongside main in scheduled-tier schedule - insta snapshot tests covering local, github, and ado emitted trees - Codecov upload (GH) and PublishCodeCoverageResults@2 (ADO) wired in Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Implementation plan 0001 phases P0-P6 P0 docs sweep, then: P1 (C2-C5): drop ox-check-tools-install-missing from the design; Propose decision bumps manifest L to the new template; purge stale manifest entries on update; categorized dry-run summary (Will create / Will update / Will propose / etc.) P2 (C6-C8): tool-version policy -- per-tool minimums + version-aware require check + rustc/pwsh gate P3 (C9-C11): three-tier impact scoping (modified / affected / required) via cargo-delta, with per-tier env vars consumed by recipes P4 (C12-C14): llvm-cov HTML report + missing lints; BASE_REF master fallback; per-recipe _ox-check-require fan-out P5 (C15-C16): GitHub actions/cache in setup composite; ADO Cache@2 in setup step P6 (C17-C18): fixture scenarios under tests/fixtures/; rewrite verification.md fixture/schema sections Also: required-tier env var introduced and used by checks classified as required (workspace-wide tools that consume the dep closure). Compile- sensitive groups (pr-fast, pr-test, pr-runtime-analysis, advisories) expanded to cross-OS. ARM64 legs added to GitHub default matrix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Cross-OS matrix work and ADO job.yml extensibility wrapper - Pre-dogfood fixes: bash shell default, arch in cache key - cargo-delta CLI / JSON shape adjustments after surveying real tool versions - Per-recipe bash shebang annotation; drop intrusive `set shell` - Initial dogfood pass on this repo + follow-up lint/rustfmt resolution - Switch recipe interpreter from bash shebang to [script("pwsh")] - Normalize line endings before hashing; lowercase justfile path migration - Cohabit-outside-region pattern for ox-tools workspace lint extras - Bump rustfmt max_width to 140 - File/region removal: handle catalog drops and disabled backends - setup-action: install libclang on Linux; install rustfmt/clippy components - InSync refreshes manifest L so stale-L from older binary versions self-heals - Fix cargo-mutants --in-diff usage; tolerate per-tool install failures - Matrix-input plumbing: hardcoded OS matrices, per-leg runner labels only (rejected fromJSON(inputs.X) form -- silent empty-matrix failure mode) - ADO backend: per-job wrapper template (steps/job.yml) for 1ESPT extensibility -- adopters edit the wrapper to plug in templateContext blocks; the dirty-file flow then preserves their edits Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Dogfood-surfaced fixes: cache, binstall, Justfile case, spellcheck - Two Linux-only CI failures (PowerShell array-unwrap, cargo-delta name format mismatch for hyphenated packages) - setup-action: save cache even on job failure (deprecated save-always replacement) - Optional cargo-binstall installer with per-tool cargo-install fallback - License header exclusions for vendored files - Per-job cache keys + drop --workspace from more recipe tools - Skip re-install when tool already present at sufficient version - Justfile case fixes: lowercase justfile -> canonical "Justfile" with case-aware migration - Cache cargo's installed-tools registry (.crates.toml) too - doc2readme: add --workspace + regenerate READMEs - Spellcheck infrastructure: preprocess .spelling -> .dic, pass `--cfg spellcheck.toml`, skip llvm-cov on Windows ARM, mkdir target/coverage - Dictionary buildup: technical jargon, lowercase variants for case-sensitive merge, intra-doc link backticking Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Per-check tuning: deny, udeps, semver-check, external-types, aprz - Trigger CI with conventional-commits PR title check - ox-check-deny: dedupe [licenses] + bump unmaintained syntax - ox-check-udeps: drop unused deps surfaced by cargo-udeps - ox-check-semver-check: filter bin-only crates from --package; tolerate unpublished + bin->lib baselines - ox-check-external-types: per-manifest invocation (no --package); invoke via cargo +nightly; tolerate rustdoc JSON schema drift - ox-check-aprz: use 'deps' subcommand, not 'check' - setup: install nightly toolchain explicitly - Test fixes for ox-check mutation-testing coverage - Iterate group placement: try moving external-types and udeps to scheduled-advisories for speed, then revert udeps/semver-check to pr-fast (acceptable PR-tier cost) - GitHub: pass GITHUB_TOKEN to per-group composite action for cargo-aprz Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Catalog tightening: pinned nightly toolchains + lint catalog reorganization - Pin nightly toolchains via versions.just; drop schema-mismatch tolerance (the pin makes drift impossible) - Ship default spellcheck.toml as a managed-region file - Switch ox-check-fmt to pinned nightly rustfmt with unstable opinions; bulk-format the codebase with that pin - Tighten ox-check lint catalog: Bucket A folding + contested-lint removals; drop empty_structs_with_brackets cohabit lint - Add rust.unexpected_cfgs to lint catalog (coverage cfgs) - Ship managed-region clippy.toml; attribute wildcard_imports bug workaround - Bump rust_nightly pin to nightly-2026-02-10 to match oxidizer Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ADO impact bootstrap + per-OS impact stages + system-deps probe - Strip pre-existing spellcheck.toml content that collides with managed region - Fix stale snapshots + spellcheck typos from recent commits - ADO impact.yml needs setup.yml to bootstrap cargo - Cross-distro libclang install in setup.yml - Env-var form for impact baseline ref selection - Skip LFS smudge in impact baseline worktree - Validate impact package names; drop unknowns - Manifest mutation tests (double-newline guard, loop-bound progress) - Unify semver-check workspace path with per-package tolerance - Add ox-check-system-deps-check: probe for system-level libs (libclang) with per-OS install hints - Rename "nightly" tier to "scheduled" (PR #25 review feedback) - Per-OS impact stages to fix cfg-conditional dep gaps (impact-linux + impact-windows; each downstream leg consumes its OS's impact set) - Local-vs-CI parity for readme-check + impact validation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Recipe TODO cleanup + cargo-heather README + advisory PR comments - cargo-heather: clarify ox-gen-readme opt-out reason; generate README via doc2readme so the README stays in sync with the crate-level rustdoc - Recipe-level cleanup: TODO on every OX_CHECK_INCLUDE recipe to factor out the preamble; inline `?? --workspace` splat; prune TODOs that turn out to be misplaced - Advisory PR comments for non-blocking findings (semver-check). Recipes exit 0 and write a markdown body to target/ox-check/comments/<NAME>.md when there are findings, remove the file when clean. GH wiring uses marocchino/sticky-pull-request-comment; ADO wiring uses a pwsh step against the ADO REST API. Skipped silently on builds without Contribute-to-PRs permissions - Extend spellcheck tokenization_splitchars to cover typographic punctuation (em-dash, en-dash, arrows) so prose with em-dashes tokenises correctly across cargo-spellcheck versions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * PR-tier reorg: unified setup, exact-pin, miri/careful + pr-slow split - Close two methodology gaps vs legacy CI (udeps --all-targets behaviour, external-types pinned-nightly enables PR-tier inclusion) - Rename pr-mutants -> pr-slow; add miri + careful to PR tier - Unified ox-check-setup recipe; install rustup components alongside cargo tools - Pin exact tool versions on install; accept newer if already present - Address real bugs surfaced by ox-check (walkdir unused, external-types API leaks: clap_builder::Error in cli, ohno::AppError in automation::run_cargo) - Bump cargo-heather pin (0.1.0 -> 0.2.1) since 0.1.0 fails to build from source on current Rust - miri exclusions for FS-heavy tests; pass `--no-tests=pass` to miri nextest only (not llvm-cov) - Add mermaid diagrams to checks.md / github.md / ado.md PR-slow CI structure: - Merge pr-test into pr-slow with three sub-recipes (slow1=tests+coverage, slow2=miri+careful, slow3=mutants) - Split pr-slow into three parallel CI jobs/stages (was: one sequential job per OS leg). Per-leg wall-clock drops from sum(slow1,slow2,slow3) to max(slow1,slow2,slow3). Umbrella `ox-check-pr-slow` just recipe preserved for local convenience CI diagram iteration: per-pipeline diagrams (PR + scheduled per backend), LR layout, single vertical column for jobs, multiple subgraph/column/ELK experiments converging on plain flowchart LR without invisible chains. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Diagram polish + crate restructure (cargo-ox-check-update) Diagram polish: - Spell out matrix legs (linux, windows, linux-arm, windows-arm) instead of "4-leg matrix"; drop tool lists from job labels (recipe column is source of truth) - Promote coverage upload to first-class node (codecov action / ADO PublishCodeCoverageResults@2) - Coverage upload: every leg except windows-arm (not linux-only). Single-leg coverage misses OS/arch-gated code; codecov coalesces multiple uploads against the same commit - Per-caller setup nodes so the ox-check-setup invocation is visible in every per-group composite/step template chain - Drop self-skip notes on pr-mutants node labels; drop verbose legends and prose paragraphs; simplify external-action labels - Add cargo delta as recipe node under impact; GH ox-check-impact now goes through ox-check-setup so cargo-delta picks up the catalog-pinned version (matches ADO behaviour) Naming reorg: - Rename pr-slow{1,2,3} to pr-{test,runtime-analysis,mutants}; rename ox-check-mutants check to ox-check-mutants-diff (parallels existing ox-check-mutants-full) - Rename cargo-ox-check -> cargo-ox-check-update; flatten the CLI (drop the redundant `update` subcommand since update is the only action) - Rename crate dir crates/cargo_ox_check_update -> crates/cargo-ox-check-update to match the hyphen convention used by cargo-coverage-gate and cargo-heather checks.md diagram: break out per-check nodes into a third layer, surface the ox-check alias, add ox-check- prefix to group labels, surface the pr-slow umbrella as a sibling of pr-fast. Other: - Drop misleading "oxidizer-github also uploads from a single canonical leg" comments (oxidizer-github actually uploads from three legs); the one-leg shape was an ox-check design simplification, then reverted - Drop orphan job.yml node from ADO PR diagram (its dotted "wraps via" edges were removed earlier and the node had no remaining edges) - Tighten diagram spacing via flowchart init config (nodeSpacing, rankSpacing, padding); bump font size on GH/ADO diagrams - Normalize crates/automation/Cargo.toml ox-check-lints managed region (drop trailing blanks that had caused a silent opt-out) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Refactor tools setup into layered recipes Replace the monolithic ox-check-setup + tool-minimums.txt / rustup-components.txt data files with a per-tool, per-component, per-toolchain install + validate-prereqs recipe layer in tools.just, composed into per-check, per-group, per-tier, and global setup recipes. All version pins now live as plain just variables in versions.just (single source of truth, no sidecar data files). CI: GH composite ox-check-setup and ADO setup.yml take a 'group' input / parameter; per-group composites pass their own group name so a pr-fast leg never installs cargo-mutants. Special value 'none' skips tool install entirely (used by impact, which only needs cargo-delta). Naming: ox-check-tool-<bin>-install, ox-check-component-<tc>-<c>-install, ox-check-toolchain-<sym>-install at the atomic layer; ox-check-<check|group|tier>-setup at composition layers. Mirror ox-check-<x>-validate-prereqs recipes at every layer. Each check recipe depends on its matching -validate-prereqs (no more _ox-check-require). Failure policy: no tolerance -- a failed install fails the recipe (cargo-mutants ARM-Windows self-skip lives in the install body). Docs: local.md, github.md, ado.md, design.md updated to match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump cargo-audit to 0.22.2 (CVSS 4.0 support) RUSTSEC-2026-0035 (pingora-cache) uses CVSS 4.0; cargo-audit 0.21.0 fails to parse the advisory database with 'unsupported CVSS version: 4.0'. 0.22.2 (released 2026-06-05) adds CVSS 4.0 support. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Probe libclang on ARM Linux multiarch path ox-check-system-deps-check only looked under /usr/lib/x86_64-linux-gnu for libclang. On ubuntu-24.04-arm runners the matching path is /usr/lib/aarch64-linux-gnu/, so the probe reported libclang missing even after apt-get install libclang-dev had put it there. Add aarch64-linux-gnu to the explicit-probe list and to the version-suffix glob fallback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump cargo-spellcheck to 0.15.7 and dedupe miri cfg_attr cargo-spellcheck 0.15.1 no longer builds from source on Rust 1.93 (the locked ra_ap_stdx triggers E0804 'cannot add auto trait Send to dyn bound via pointer cast'), which broke the linux-arm pr-fast leg where binstall has no ARM prebuilt and falls back to source. Bump to 0.15.7; the em-dash word-boundary regression that motivated the original 0.15.1 pin is already mitigated by the typographic-punctuation entries we appended to tokenization_splitchars in spellcheck.toml's managed region (Phase 7.5). Separately, six tests in cargo-coverage-gate's workspace module stacked two #[cfg_attr(miri, ignore = ...)] attributes per test (one citing filesystem use, one citing cargo metadata subprocess). Newer rustc (nightly used by extended-analysis) treats the second ignore as redundant and emits 'unused attribute' under -D warnings, failing the legacy extended-analysis workflow. Collapsed each pair into a single attribute with a combined reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Dedupe one more miri cfg_attr in lcov_cov.rs Same pattern as the workspace.rs fix: from_path_reads_from_disk had two stacked #[cfg_attr(miri, ignore = ...)] attributes (lines 166 and 168 straddling the #[test] attribute). Newer rustc emits 'unused attribute' on the second one. Collapsed into one with a combined reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename cargo-ox-check-update to cargo-anvil Full rebrand of the unified-builds tool from cargo-ox-check-update / ox-check-* to cargo-anvil / anvil-*. Scope: - Crate / binary: crates/cargo-ox-check-update -> crates/cargo-anvil; Cargo name 'cargo-ox-check-update' -> 'cargo-anvil'; clap bin_name 'cargo ox-check-update' -> 'cargo anvil'. The subcommand token cargo injects is now 'anvil' and parse_from_cargo_args strips that. - Just recipe namespace: every ox-check-* recipe (groups, tiers, per-check, per-tool *-install / *-setup / *-validate-prereqs) renamed to anvil-*. 'just anvil' aliases anvil-pr. - Directories: justfiles/ox-check/ -> justfiles/anvil/ (both workspace and templates); .github/actions/ox-check-* -> .github/actions/anvil-*; .github/workflows/ox-check-pr*.yml + ox-check-scheduled*.yml renamed to anvil-pr*.yml + anvil-scheduled*.yml. - Lockfile: .ox-check.lock -> .anvil.lock. - Env vars: OX_CHECK_* -> ANVIL_*; ADO pipeline variable ox_check_rustc_version -> anvil_rustc_version. - Managed-region marker strings: '# managed by ox-check' -> '# managed by cargo anvil' (visible to every consuming repo on next regen). - Docs / README: design/, implementation-plans/, verification.md, all templates, snapshots regenerated. Verified locally: cargo check + 199 lib tests + 4 schema tests + 3 snapshot tests pass; 'cargo anvil' is idempotent on this workspace (31 items, 0 changes); 'just format' clean. The tests/update.rs integration binary triggers a Windows local-only UAC heuristic on the developer machine (filename starts with 'update'); CI runs it fine. Skipping pr-fast locally because cargo-aprz network appraisal runs >30min; CI is faster. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add naming rationale to design.md The rename to cargo-anvil is a deliberate brand choice (vs. the prior descriptive cargo-ox-check-update name). Add a one-paragraph note at the top of design.md explaining the forge metaphor, isolated so it doesn't leak into the user-facing surfaces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Use cargo-bins/cargo-binstall action in setup-action.yml Install a prebuilt cargo-binstall binary up front via the official action so the bootstrap branch in _install-tool (which runs 'cargo install --locked cargo-binstall' from source, ~4 min per cold-cache job) becomes a no-op on GH. Also opportunistically use binstall for 'just' itself with cargo-install fallback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Replace 'CI' with 'cloud workflows' in cargo-anvil docs/comments Addresses Sander's review feedback on PR #33: 'CI' is a working methodology, not a synonym for automation pipelines. Adopt 'cloud workflows' (neutral across GH Actions and ADO pipelines) throughout cargo-anvil docs, source comments, template comments, and Cargo.toml description/keywords. Scope: cargo-anvil only. No code identifiers changed. Pre-existing ox-tools repo docs (top-level README.md, DEVELOPMENT.md) and other crates' READMEs left alone since they predate cargo-anvil and were not part of Sander's review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix anvil-readme-check on rebased branch Two follow-ups after rebasing on origin/main: 1. Regenerate crates/cargo-anvil/README.md — the prior CI -> cloud workflows rename in 71ad75c edited lib.rs doc-comments but didn't re-run doc2readme, so the README drifted. 2. Opt cargo-heather out of anvil-readme-check via [package.metadata.ox-gen-readme] disable = true. cargo-heather ships a hand-curated README with its own logo and layout and is already excluded from the legacy 'just readme'/'readme-check' recipes via hardcoded '--ignore cargo-heather'. anvil-readme-check supports the per-crate opt-out documented at checks.just:215. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Regenerate cargo-heather README (drift from v0.2.1 -> v0.3.0 bump) Reverts the [package.metadata.ox-gen-readme] disable opt-out added in 8639a43 (incorrect: 7a3dcd4 on main intentionally moved cargo-heather to the shared doc2readme pipeline). The real root cause was stale docs.rs URLs in README.md: PR #40 bumped the crate to v0.3.0 but didn't regenerate the README. main's legacy 'just readme-check' skips cargo-heather via hardcoded --ignore cargo-heather so the drift went undetected; anvil-readme-check (which has no per-crate skip-list) correctly flagged it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

martin-kolinek added 4 commits May 13, 2026 14:47

Initial design doc on cargo-ox-ci

c9c92d3

Split design do

2bae517

Include top-level pipelines

3c2827f

Better updates

a63a1f6

martin-kolinek changed the title ~~Design: cargo-ox-ci, unified Rust build/CI scaffolding tool~~ chore: cargo-ox-ci, unified Rust build/CI scaffolding tool May 13, 2026

martin-kolinek and others added 3 commits May 13, 2026 20:57

Sander Saares (sandersaares) reviewed May 14, 2026

View reviewed changes

martin-kolinek changed the title ~~chore: cargo-ox-ci, unified Rust build/CI scaffolding tool~~ chore: cargo-ox-check, unified Rust build/CI scaffolding tool May 14, 2026

Sander Saares (sandersaares) reviewed Jun 2, 2026

View reviewed changes

martin-kolinek closed this Jun 4, 2026

Uh oh!

Conversation

martin-kolinek commented May 13, 2026

Design summary

Key design choices

Status

Uh oh!

Codecov Comments Bot (codecov-commenter) commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Sander Saares (sandersaares) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Sander Saares (sandersaares) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

martin-kolinek commented Jun 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Codecov Comments Bot (codecov-commenter) commented May 13, 2026 •

edited

Loading