Skip to content

Commit fd876a7

Browse files
utofutof
authored
Merge pull request #145 from utof/test-architecture/v0.6.x
test architecture v0.6.x — T1 (mutants + coverage + fuzz + criterion)
2 parents df93f00 + a19e76d commit fd876a7

19 files changed

Lines changed: 3866 additions & 387 deletions

File tree

‎.cargo/mutants.toml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# cargo-mutants config — see https://mutants.rs/configuration
2+
#
3+
# WHY this file exists: per-PR mutation testing wired as a CI workflow
4+
# (.github/workflows/mutants.yml) needs centralised scope + test-runner
5+
# config that survives independent of the workflow file. See spec
6+
# docs/superpowers/specs/2026-04-24-cargo-mutants-design.md.
7+
8+
# WHY test_tool = "nextest" (mandatory): CLAUDE.md "Test stack" bans
9+
# `cargo test` for non-doc tests because of the SQLite lock-order
10+
# inversion deadlock (GH #131). cargo-mutants defaults to `cargo test`;
11+
# this override routes through `cargo nextest` instead. The repo-level
12+
# scripts/no-cargo-test.sh enforces the ban only on yml/sh/justfile
13+
# string scans, so cargo-mutants invoked via `cargo mutants` would NOT
14+
# be caught — but the deadlock still hits, hence the override is
15+
# load-bearing for correctness.
16+
test_tool = "nextest"
17+
18+
# WHY exclude_globs targets crates/desktop/**: Tauri/GTK system-dep
19+
# friction (see Batch D known env limitation in
20+
# docs/superpowers/plans/architecture-audit-done-cheatsheet.md). Most
21+
# desktop code is wire-up; mutation surface is low value. Re-introducing
22+
# desktop coverage requires a separate Linux-deps setup commit + a
23+
# justification for the CPU cost. This is the SOLE mechanism scoping
24+
# mutation generation — do NOT also pass `--workspace --exclude
25+
# perima-desktop` via `additional_cargo_args` (that key is for build
26+
# flags like `--release`/`--all-features` and would clash with
27+
# cargo-mutants' own per-package selection layer).
28+
exclude_globs = ["crates/desktop/**"]
29+
30+
# WHY timeout_multiplier = 5.0: ScanUseCase + WriteCmd dispatch are
31+
# fundamentally async + spawned tasks; cargo-mutants' default per-mutant
32+
# timeout heuristics over-trigger on async fns. 5x default ≈ 25s per
33+
# mutant kill attempt before cargo-mutants logs `TIMEOUT`. Tune down
34+
# once baseline is observed across ~10 PRs.
35+
timeout_multiplier = 5.0

‎.github/workflows/bench.yml‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# criterion benchmarks weekly cron + manual workflow_dispatch.
2+
# Slice 4 of T1 test-architecture decomposition.
3+
# See spec: docs/superpowers/specs/2026-04-25-criterion-benches-design.md
4+
#
5+
# WHY a separate workflow (not added to ci.yml): criterion runs are
6+
# bounded MINUTES, not seconds; per-PR queue cost is disproportionate.
7+
# Decoupling lets bench fail-or-skip independently of the main matrix.
8+
9+
name: bench
10+
11+
on:
12+
schedule:
13+
- cron: '0 7 * * 1' # Mondays 07:00 UTC (1h after slice 3 cargo-fuzz)
14+
workflow_dispatch:
15+
inputs:
16+
sample_size:
17+
description: 'criterion sample size per benchmark'
18+
type: string
19+
default: '30'
20+
21+
permissions:
22+
contents: read
23+
24+
env:
25+
CARGO_TERM_COLOR: always
26+
27+
jobs:
28+
bench:
29+
name: cargo bench
30+
runs-on: ubuntu-latest
31+
steps:
32+
- uses: actions/checkout@v4
33+
34+
- uses: dtolnay/rust-toolchain@stable
35+
- uses: Swatinem/rust-cache@v2
36+
37+
# WHY GTK deps even though no benched crate uses them: matches
38+
# slice 1/3 workflow-scaffolding pattern. perima-hash + perima-db
39+
# don't transitively need GTK; the install is defensive.
40+
- name: Install image/system deps
41+
run: |
42+
sudo apt-get update -q
43+
sudo apt-get install -yq \
44+
libgtk-3-dev \
45+
libwebkit2gtk-4.1-dev \
46+
libayatana-appindicator3-dev \
47+
librsvg2-dev \
48+
patchelf
49+
50+
# WHY continue-on-error: per spec D-4, observability-only mode.
51+
# Bench step might fail on transient runner issues; swallowing
52+
# the exit lets artifact upload still run. Consistent posture
53+
# with slice 1/3 workflows.
54+
#
55+
# WHY scoped to `-p perima-hash -p perima-db` (NOT --workspace):
56+
# this slice pins exactly 2 benched crates. `cargo bench --workspace`
57+
# would compile + invoke benches in any future workspace crate that
58+
# gets a [[bench]] entry, scope-creeping the slice's contract.
59+
- name: Run cargo bench (perima-hash + perima-db)
60+
continue-on-error: true
61+
env:
62+
SAMPLE_SIZE: ${{ github.event.inputs.sample_size || '30' }}
63+
run: |
64+
# Tee bench output to GITHUB_STEP_SUMMARY so the run page shows
65+
# the latest numbers without artifact download.
66+
{
67+
echo "## criterion benchmark results"
68+
echo ""
69+
echo '```'
70+
cargo bench -p perima-hash -p perima-db -- --sample-size "$SAMPLE_SIZE" 2>&1
71+
echo '```'
72+
} | tee -a "$GITHUB_STEP_SUMMARY"
73+
74+
- name: Upload criterion target dir (raw HTML reports)
75+
if: always()
76+
uses: actions/upload-artifact@v4
77+
with:
78+
name: criterion-reports
79+
path: target/criterion/
80+
if-no-files-found: ignore

‎.github/workflows/fuzz.yml‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# cargo-fuzz weekly cron + manual workflow_dispatch.
2+
# Slice 3 of T1 test-architecture decomposition.
3+
# See spec: docs/superpowers/specs/2026-04-25-cargo-fuzz-design.md
4+
#
5+
# WHY a separate workflow (not added to ci.yml): fuzz runs are bounded
6+
# minutes, NOT seconds; they would dominate per-PR queue if added to
7+
# the main matrix. Decoupling lets fuzz fail-or-skip independently of
8+
# the main `just ci` matrix.
9+
10+
name: fuzz
11+
12+
on:
13+
schedule:
14+
- cron: '0 6 * * 1' # Mondays 06:00 UTC
15+
workflow_dispatch:
16+
inputs:
17+
max_total_time:
18+
description: 'Per-target fuzz duration in seconds'
19+
type: string
20+
default: '300'
21+
22+
permissions:
23+
contents: read
24+
25+
env:
26+
CARGO_TERM_COLOR: always
27+
28+
jobs:
29+
fuzz:
30+
name: cargo fuzz / ${{ matrix.target }}
31+
runs-on: ubuntu-latest
32+
strategy:
33+
fail-fast: false
34+
matrix:
35+
target: [exif, blake3]
36+
steps:
37+
- uses: actions/checkout@v4
38+
39+
# WHY install nightly here (not via dtolnay/rust-toolchain@stable):
40+
# the fuzz/ subdir's rust-toolchain.toml pins nightly; rustup honors
41+
# the toolchain file when invoked from inside fuzz/.
42+
- name: Install nightly via fuzz/rust-toolchain.toml
43+
run: |
44+
cd fuzz
45+
rustup show
46+
47+
- uses: Swatinem/rust-cache@v2
48+
with:
49+
workspaces: fuzz -> target
50+
51+
- name: Install cargo-fuzz
52+
uses: taiki-e/install-action@v2
53+
with:
54+
tool: cargo-fuzz
55+
56+
# WHY install GTK deps defensively: matches slice 1 cargo-mutants
57+
# workflow pattern to keep workflow scaffolding uniform across the
58+
# T1 test-architecture slices. Strictly speaking, perima-hash +
59+
# perima-media do NOT transitively require GTK on ubuntu-latest's
60+
# default features; keeping the install avoids silent breakage if
61+
# a future cargo-features change pulls in something system-dep-ful.
62+
- name: Install image/system deps
63+
run: |
64+
sudo apt-get update -q
65+
sudo apt-get install -yq \
66+
libgtk-3-dev \
67+
libwebkit2gtk-4.1-dev \
68+
libayatana-appindicator3-dev \
69+
librsvg2-dev \
70+
patchelf
71+
72+
# WHY continue-on-error: per spec D-4, observability-only mode.
73+
# libFuzzer exits non-zero on a crash; without continue-on-error
74+
# the very first crash blocks all future runs of this workflow
75+
# until the crash is fixed. Precedent: kani.yml + mutants.yml use
76+
# the same pattern.
77+
- name: Run cargo fuzz target ${{ matrix.target }}
78+
continue-on-error: true
79+
env:
80+
MAX_TIME: ${{ github.event.inputs.max_total_time || '300' }}
81+
run: |
82+
cd fuzz
83+
cargo fuzz run ${{ matrix.target }} -- -max_total_time=$MAX_TIME
84+
85+
- name: Upload artifacts (if any crashes)
86+
if: always()
87+
uses: actions/upload-artifact@v4
88+
with:
89+
name: fuzz-artifacts-${{ matrix.target }}
90+
path: fuzz/artifacts/${{ matrix.target }}/
91+
if-no-files-found: ignore

‎.github/workflows/mutants.yml‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# cargo-mutants per-PR mutation testing — Linux-only, observability-only.
2+
# See spec: docs/superpowers/specs/2026-04-24-cargo-mutants-design.md
3+
#
4+
# WHY a separate workflow (not added to ci.yml's matrix): mutation
5+
# testing is CPU-heavy + slow (minutes per file); decoupling lets it
6+
# fail-or-skip independently of the main `just ci` matrix. PR contributors
7+
# see two checks instead of one. Failure of mutants.yml does NOT block
8+
# merge during slice 1 (continue-on-error swallows non-zero exit).
9+
10+
name: mutants
11+
on:
12+
pull_request:
13+
14+
permissions:
15+
contents: read
16+
17+
env:
18+
CARGO_TERM_COLOR: always
19+
20+
jobs:
21+
mutants:
22+
name: cargo mutants --in-diff
23+
runs-on: ubuntu-latest
24+
if: github.event_name == 'pull_request'
25+
steps:
26+
- uses: actions/checkout@v4
27+
with:
28+
# WHY fetch-depth: 0 — `git diff origin/<base>..` requires the
29+
# base branch in local history; depth 0 fetches the full graph.
30+
fetch-depth: 0
31+
32+
- uses: dtolnay/rust-toolchain@stable
33+
- uses: Swatinem/rust-cache@v2
34+
35+
# WHY taiki-e/install-action (NOT cargo-binstall): this workflow
36+
# runs on ubuntu-latest only; the BASH_FUNC_* leak that broke
37+
# taiki-e/install-action on windows-latest (GH #137) does NOT
38+
# affect Linux. Multi-tool comma syntax fetches both binaries in
39+
# one step.
40+
- uses: taiki-e/install-action@v2
41+
with:
42+
tool: cargo-mutants,cargo-nextest
43+
44+
# WHY Tauri Linux deps even though crates/desktop is excluded from
45+
# mutation: `cargo metadata` (which cargo-mutants invokes
46+
# internally) resolves the WHOLE workspace including
47+
# perima-desktop's transitive deps. Without GTK headers, metadata
48+
# resolution fails before any mutation happens.
49+
- name: Install Tauri Linux deps
50+
run: |
51+
sudo apt-get update -q
52+
sudo apt-get install -yq \
53+
libgtk-3-dev \
54+
libwebkit2gtk-4.1-dev \
55+
libayatana-appindicator3-dev \
56+
librsvg2-dev \
57+
patchelf
58+
59+
# WHY `| tee git.diff` (vs `> git.diff`): tee echoes the diff into
60+
# the CI log so reviewers can see what cargo-mutants will operate
61+
# on without downloading the artifact. Matches the canonical
62+
# mutants.rs/in-diff example.
63+
- name: Compute diff against base branch
64+
run: git diff origin/${{ github.base_ref }}.. | tee git.diff
65+
66+
# WHY continue-on-error: per spec D-4, this slice is observability-
67+
# only. cargo-mutants exits 2 when mutants survive, 3 on timeout,
68+
# 4 on baseline failure. Without continue-on-error the very first
69+
# PR with a survivor blocks the workflow + defeats the
70+
# observability-first calibration goal. Precedent: kani.yml uses
71+
# the same pattern. Drop this in a future slice once the survivor
72+
# baseline is stable.
73+
- name: cargo mutants --in-diff
74+
continue-on-error: true
75+
run: cargo mutants --no-shuffle -vV --in-diff git.diff
76+
77+
- name: Upload mutants.out
78+
if: always()
79+
uses: actions/upload-artifact@v4
80+
with:
81+
name: mutants-out
82+
path: mutants.out

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,10 @@
1212
# .github/; the blanket **/*.md rule would otherwise exclude markdown
1313
# files added there (e.g. pull_request_template.md).
1414
!.github/**
15+
# WHY: fuzz/README.md is a contributor-facing quick-start doc for the
16+
# cargo-fuzz harness; it needs to be tracked so new contributors can
17+
# find invocation + triage instructions without reading source files.
18+
!fuzz/README.md
1519
target/
1620
.DS_Store
1721
*.swp

0 commit comments

Comments
 (0)