Commit 6594bf5
chore(skills): add consistency-auditor agent + readme-walkthrough hook for Phase A (#382)
* chore(skills): port consistency-auditor + readme-walkthrough hook from vuls-reach
Back-port of the Phase A claim-consistency layer from vuls-reach PR #25
(Layer B+C). Adds a 6th conditional Phase A reviewer that catches the
"narrative drift" failure mode where the same factual claim (version
range, package name, license expression, command flag, manifest header)
appears in multiple files within a single PR with different values —
the failure shape that drives multi-round Copilot ↔ Claude review
cascades on doc-heavy PRs.
## What's added
- **`.claude/agents/consistency-auditor.md`** — 6th named subagent for
Phase A. Detects cross-file fact drift (8 claim classes: version-range,
identifier-literal, command-walkthrough, filename-pattern, schema-column,
fix-mechanism-narrative, command-flag, license-expression). Pre-reads
`.github/instructions/copilot-learned-coding.instructions.md` for the
canonical rule set. 5-step procedure: build fact map -> cross-reference
within PR -> cross-reference unchanged files -> walkthrough invariant
check -> fix-diff content integrity.
- **`.claude/hooks/readme-walkthrough.sh`** — pre-push static check.
Awk-based scanner walks fenced bash/sh/shell/zsh/console blocks in
changed `*README*.md` and `docs/*.md`, flags blocks that mix
repo-root-relative paths (`cmd/`, `internal/`, `pkg/`, `examples/`,
`bin/`, ...) with `cd <name>` fixture-relative refs. CRLF-safe;
non-blocking `additionalContext` output. root_re prefixes are
`cmd|internal|pkg|examples|scripts|testdata|third_party|claude-skills|bin|docs|.github|.claude`,
bare-file alternation includes `Makefile|go.mod|go.sum|.golangci.yml|uzomuzo|uzomuzo-diet`.
- **`.claude/settings.json`** — wires `readme-walkthrough.sh` into the
PreToolUse Bash matcher gated on `git push`, after `pre-push-review.sh`.
- **`.github/prompts/review-until-clean.prompt.md`** — Phase A becomes
**5 or 6 agents** with a pre-filter: Agent 6 (consistency-auditor)
only spawns for PRs that touch `*.md` / `*.txt` / `*.tsv` / `*_test.go`
/ `testdata/**` / `internal/testdata/**` (claim-bearing files). Pure
non-test Go PRs save the spawn cost. NEW **Step 1.5** generates a fact
map (4-column schema: Class, Fact key, Value, Asserted in) for
doc-heavy PRs that consistency-auditor consumes as its baseline. BASE
is captured once at Step 1 so Steps 1.5 / 2 / 4 share the same diff
anchor. Stop-condition table updated to reflect the configured agent
count.
## Adaptations vs. vuls-reach
- **Domain-generalized claim classes**: removed vuls-reach-specific
examples (FunctionID, snakeyaml-cve-2017-18640) in favor of OSS
scanning examples; added `license-expression` class (uzomuzo-oss-
relevant) on top of the seven inherited classes.
- **Clone-and-cd exception**: when a block contains `git clone` followed
by `cd <name>`, the cd is recognized as a repo-root navigation (not
fixture-relative). Avoids false positives on the standard "Build from
source" walkthrough at README.md:96.
- **`./<name>` no longer triggers fixture signal**: only an explicit
`cd <name>` changes shell CWD and is therefore the only signal of
fixture-relative state. Bare `./<name>` path arguments
(`trivy fs ./my-project`, `./uzomuzo scan`) are command arguments,
not CWD changes — flagging them produced false positives on every
README example that names a path.
- **Skip the replay fixture**: vuls-reach ships
`internal/testdata/review-cascade/snakeyaml-pr20/` as acceptance
criterion. uzomuzo-oss's domain has no equivalent corpus PR cascade
to replay; the agent works without the fixture (fact-map mode +
cross-reference). A uzomuzo-relevant fixture is a follow-up.
## Why now
uzomuzo-oss already has Layer A (the `.github/instructions/` SoT files
and `make sync-instructions` mirror generation), but lacks the *active*
detectors that catch new drift before Phase B. Layer B+C is the
narrowest, most clearly-portable improvement in vuls-reach PR #25; it
has no Go-tool dependency and no CI workflow change, so the back-port
is contained to four files.
Layer A activation (Phase D in the prompt + `internal/devtools/sync-instructions/`
Go tool + `sync-instructions-freshness.yml` CI gate) is a separate scope.
## Verification
- `bash -n .claude/hooks/readme-walkthrough.sh` passes.
- `jq . .claude/settings.json` parses cleanly.
- Smoke test against the existing `README.md`, `claude-skills/README.md`,
`docs/*.md` produces zero false positives after the fixture_re
simplification.
- Synthetic mixed-CWD test
(`cd vulnerable && mvn package` followed by `tools/...`) is detected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address Copilot review on PR #382 (round 2)
Update doc-code drift in `.claude/hooks/readme-walkthrough.sh`:
- Header comment (line 4-8): old text described the heuristic as flagging
blocks that mix repo-root paths "with `cd <name>` / `./<name>`
fixture-relative refs". The implementation no longer treats bare
`./<name>` path arguments as fixture-relative — only `cd <name>` is a
true CWD change. Updated to `cd <name>` only and added a note that
`./<name>` arguments do NOT trigger the signal.
- `scan_blocks` comment (line 55): same drift, same fix.
- User-facing warning printf (line 124): the warning text shown to
developers when the hook fires said "fixture-relative (cd <name>,
./<name>) paths". Updated to "a `cd <name>` fixture-relative shell
CWD change" so developers see the actual heuristic.
The two `WONT_FIX` Copilot threads on this PR (line 47, line 194) are
about `docs/*.md` pathspec recursion. Empirical test (with a fresh
git repo containing `docs/top.md`, `docs/adr/foo.md`,
`docs/case-studies/bar.md`):
$ git ls-files -- 'docs/*.md'
docs/adr/README.md
docs/adr/foo.md
docs/case-studies/bar.md
docs/top.md
confirms `docs/*.md` matches nested files (git's default pathspec uses
fnmatch where `*` matches `/`). The script's existing comment on the
DOC_FILES line ("`*` matches any character INCLUDING `/`") is correct;
no fix needed. Phase C will reply WONT_FIX with the test evidence.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: pre-empt mirror-provenance drift in prompt (PR #382 round 3)
The Phase A Step 2 paragraph said `.claude/rules/agents.md` is
"hand-curated", but in this repo that file is generated from
`.github/instructions/agent-orchestration.instructions.md` (carries the
canonical "DO NOT EDIT DIRECTLY" mirror header from `make sync-instructions`).
Reframed the paragraph: `consistency-auditor` is invoked by file
presence at `.claude/agents/consistency-auditor.md` regardless of any
registry, so the generated `agents.md` mirror does not need to list it
for the skill to work. The instruction-file SoT update is a follow-up,
independent of this skill working.
Same correction was made to the catalog port (uzomuzo-catalog PR #227
round 2) where Copilot caught the same drift; applying it here
proactively before Copilot's next review pass on the same paragraph.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: instruction-sync compliance and comment-code consistency (PR #382 Phase A)
- Convert .claude/agents/consistency-auditor.md to thin delegation shim
- Create .github/agents/consistency-auditor.agent.md as canonical source
- Fix phantom rule references ("Update Cross-File Facts Atomically",
"Documentation & Narrative Consistency") → "Comment-Code Consistency"
- Fix "claim triples" → "claim records" (schema is 4-field tuples)
- Add sh-session/shell-session to hook header and scan_blocks comments
(aligning comments with the fence_open regex that already matches them)
- Add "e.g.," qualifier to Agent 6 claim-class list (non-exhaustive)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove redundant gsub mask for bare ./<prefix> paths (PR #382 round 4)
The first gsub that stripped `./(repo-root-prefix)/...` tokens was dead
code: fixture_re requires a `cd` prefix, so bare `./cmd/uzomuzo` path
arguments can never match it. The comment incorrectly claimed these
tokens would be "double-counted as fixture-relative". Remove the dead
gsub and correct the comment to accurately describe the cd-only
masking heuristic.
Co-authored-by: Kota Kanbe <kotakanbe@users.noreply.github.com>
* fix: correct misleading comment about bare-file gsub rationale (PR #382 round 5)
The comment claimed `./go.mod` would match fixture_re, but fixture_re
requires a `cd` prefix — bare path arguments can never trigger it. The
actual false-positive scenario is `cd uzomuzo` (repo-root directory name
matching cd_arg's `[a-z][a-z0-9_./-]*` pattern). Corrected the example
and listed all masked tokens (the old comment omitted uzomuzo/uzomuzo-diet).
Co-authored-by: Kota Kanbe <kotakanbe@users.noreply.github.com>
* fix: narrow clone-and-cd exception to first post-clone cd only (PR #382 round 6)
The `saw_clone` flag was never cleared after consuming the post-clone
`cd <repo-name>`, so ALL subsequent `cd <single-token>` commands in the
same block were masked. A later `cd demo` (real fixture transition)
would be silently stripped, hiding a genuine mixed-CWD mismatch.
Fix: wrap the gsub in `if (gsub(...))` and clear `saw_clone = 0` after
the first post-clone cd is consumed. Later cd commands now participate
in normal fixture detection.
Co-authored-by: Kota Kanbe <kotakanbe@users.noreply.github.com>
* fix: address Copilot doc-code drift on PR #382 (round 7) — BASE/Step 4
Sister fix to uzomuzo-catalog PR #227 round 5. Two related fixes to
`.github/prompts/review-until-clean.prompt.md`:
1. **BASE/Step 4 inconsistency**: the Step 1 BASE comment (line 72-75)
and the Step 2 paragraph (line 116) both claimed BASE is reused for
"Step 4's verification", but Step 4 in this prompt is the build/
vet/test/lint step and does not reference BASE — only Step 1.5
fact-map walk and Step 2 pre-filter use it. Reworded both sites to
mention only the steps that actually use BASE.
2. **BASE fallback robustness**: the original
`git merge-base HEAD origin/main 2>/dev/null || echo main`
silently degrades to a literal `main` string if `origin/main` is
unavailable, so on a `--single-branch=feature` clone with no `main`
ref the subsequent `git diff "$BASE" HEAD` would error and
DOC_TOUCHING (the AGENT_COUNT pre-filter signal) would silently
become empty. Added a middle fallback to local `main` merge-base:
`... || git merge-base HEAD main 2>/dev/null || echo main`.
Copilot's most recent review on a8c5c75 reported "no new comments"
(literal Copilot-clean), but the user's `/review-until-clean`
re-review experiment on the sister catalog PR surfaced the same
inherited drift on prompt.md — fixing here pre-emptively to keep the
two ports consistent and to avoid Copilot flipping the catalog-side
finding onto oss in a future review pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address Copilot review on PR #382 (round 8) — pathspec comment + repro hint
Two doc-code drift fixes in `.claude/hooks/readme-walkthrough.sh`:
1. **Exclusion comment uses short form, code uses long form**
(Copilot thread `_AIrD8`): the comment described the testdata
exclusions as `:!testdata/**` / `:!internal/testdata/**` while the
actual `git diff` line used `:(exclude)testdata/**` /
`:(exclude)internal/testdata/**`. Both forms are equivalent (short
`:!` vs long `:(exclude)` git pathspec magic), but the inconsistent
doc form confuses readers trying to map the comment to the code.
Updated the comment to use the long form (matches the code) and
noted the equivalence to the short form.
2. **`additionalContext` repro hint missed exclusions** (sister thread
on catalog #227 round 6 surfaced this): the user-facing
`additionalContext` message suggested
`git diff ${BASE} HEAD -- "*README*.md" "docs/*.md"` for
reproduction, but the scanner's actual DOC_FILES pathspec also
excludes `testdata/**` and `internal/testdata/**`. Reproducers
running the suggested command would see different files than the
scanner walked. Updated the hint to include the same exclusions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>1 parent 3812a21 commit 6594bf5
5 files changed
Lines changed: 432 additions & 6 deletions
File tree
- .claude
- agents
- hooks
- .github
- agents
- prompts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
41 | 47 | | |
42 | 48 | | |
43 | 49 | | |
| |||
0 commit comments