app-ai-guardrails v0.0.3: org-preset lint source, greenfield-ideal thresholds, eslint lessons - #17
Conversation
…resholds, eslint lessons, non-interactive fix
Learnings folded from an org shared-config release and its staged consumer migration:
- Phase 0 'lint source' parameter: org preset package preferred when one exists (thin consumer:
factory + overrides only); inline canon stays the self-contained default. Scorer resolves rules
via eslint --print-config when text-grep misses, so preset-composed repos score identically
(validated: scratch consumer resolved 268 sonarjs rules, cognitive-complexity 15).
- Coverage thresholds raised to the greenfield ideal: 90 everywhere, branches 85 (per-stack
mappings in canon/coverage.md). The inherited 85/71/76/86 were one repo's brownfield ratchet
floors. Scorer checks enforcement behaviorally, so the published ledger is unaffected; golden
regenerated, self-test ALL CHECKS PASSED.
- ESLint flat-config option-retention gotcha (explicit {} on overridden rules) + anti-defang entry;
ESLint>=10 escape route (@next/eslint-plugin-next direct); .d.ts no-var override;
no-alphabetical-sort + e2e/** test relaxations; prefer-read-only-props off with rationale;
tsconfig noFallthroughCasesInSwitch (+ documented noUncheckedIndexedAccess omission).
- Non-interactive question-stall fix (v0.0.2 regression, caught by this release's confirmation
run): Phase 0 defines non-interactive and forbids asking when no reply can arrive; consent
gate's non-interactive branch is the TODO(skills-install) block. Benchmark prompt regained its
designed do-not-ask line. Confirmation: fresh sonnet run scored 94, all gates green.
- Shipped-text deferral/roadmap wording removed (no-op context per writing-great-skills).
Version 0.0.3 in frontmatter + marketplace.json.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates the app-ai-guardrails skill and its benchmark/scorer for the v0.0.3 release, incorporating lessons from an org-shared ESLint preset migration, raising “greenfield ideal” coverage floors, and hardening non-interactive behavior to avoid stalling runs.
Changes:
- Bumped
app-ai-guardrailsto v0.0.3 and synced the version in.claude-plugin/marketplace.json. - Raised coverage thresholds across canon docs, adapters, and benchmark fixtures (generally to 90%+ with stack-appropriate axes).
- Improved lint/scoring guidance and benchmark behavior: org-preset lint source path, ESLint flat-config option-retention note, “do not ask questions” benchmark prompt, and scorer support for preset-composed ESLint configs via
eslint --print-config.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| skills/app-ai-guardrails/SKILL.md | v0.0.3 bump; Phase 0 parameter/non-interactive wording updates and lint-source parameterization. |
| skills/app-ai-guardrails/references/canon/gate-interface.md | Adds an ESLint flat-config anti-pattern entry about severity-only overrides retaining prior options. |
| skills/app-ai-guardrails/references/canon/coverage.md | Updates per-stack coverage threshold table to the raised “greenfield ideal” floors. |
| skills/app-ai-guardrails/references/canon/agent-surface.md | Clarifies consent gate behavior for non-interactive runs; adjusts TODO block wording. |
| skills/app-ai-guardrails/references/adapters/springboot.md | Raises JaCoCo verification minimums and updates explanatory text. |
| skills/app-ai-guardrails/references/adapters/rust.md | Raises cargo-llvm-cov fail-under thresholds. |
| skills/app-ai-guardrails/references/adapters/next.md | Adds org-preset guidance, ESLint ≥10 escape route notes, test/e2e relaxations, d.ts overrides, and raised coverage thresholds. |
| skills/app-ai-guardrails/references/adapters/nest.md | Raises coverage thresholds and points to reusing updated Next ESLint/org-preset guidance. |
| skills/app-ai-guardrails/references/adapters/go.md | Raises Go coverage gate threshold; refines gomod-age note. |
| skills/app-ai-guardrails/references/adapters/django.md | Raises pytest-cov fail-under threshold in narrative and config block. |
| skills/app-ai-guardrails/README.md | Updates scope wording to “out of scope” (no roadmap language). |
| README.md | Updates Claude Code plugin install examples to use the renamed marketplace (@agent-skills). |
| benchmarks/app-ai-guardrails/task/prompt-template.md | Restores/strengthens “Do not ask questions” instruction to prevent stall regressions. |
| benchmarks/app-ai-guardrails/results/runs.csv | Adds additional benchmark run records for the release confirmation. |
| benchmarks/app-ai-guardrails/fixtures/make-golden.sh | Updates vitest coverage thresholds in golden fixture generation. |
| benchmarks/app-ai-guardrails/check-guardrails.mjs | Enhances Next static-analysis probes to resolve rules via eslint --print-config when using preset-composed configs. |
| .claude-plugin/marketplace.json | Renames marketplace to agent-skills and syncs app-ai-guardrails plugin version/description. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- SKILL.md: non-interactive rule now states stack precedence explicitly - stated/derivable -> proceed with defaults; missing -> abort (the one never-defaulted parameter); ask only when reachable. Resolves the proceed-vs-ask contradiction. - agent-surface.md: TODO(skills-install) commands carry -y, consistent with the install-set rule - next.md: suppression flags kept concrete but anchored to the currency rule (verify via eslint --help at use time) - run-arm.sh: hard TERM/KILL watchdog replaces SIGALRM (observed runs hanging 60-99min past the soft alarm; CLI survives SIGALRM) Behavioral note: run 14 (94, gates green) confirmed the exercised non-interactive path on this text; the edits refine only the stack-missing abort edge, which the benchmark prompt cannot exercise (it always states the stack). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ires a real file - run-arm.sh: the KILL escalation now verifies the PID still runs a claude/node command (ps comm check) - a fast TERM exit within the 30s grace can no longer kill a reused PID - check-guardrails.mjs: the --print-config fallback resolves against a real source file via the mutation-target walker instead of a fixed src/ path; no file -> keep text-grep verdicts. Fixes false negatives on non-src layouts. Self-test ALL CHECKS PASSED. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review dispositions (all rounds — surfaced here because inline replies collapse under outdated threads)Round 1 (fixed in e361d9c):
Round 2 (fixed in 2f77ec3): 🤖 Generated with Claude Code |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
skills/app-ai-guardrails/references/canon/agent-surface.md:108
- In the TODO(skills-install) block,
-yis currently placed afteradd, which passes it to theskillsCLI rather than tonpx. That can still allownpxto prompt for installing theskillspackage (defeating the non-interactive fallback) and it’s inconsistent with the earlier guidance to run installs with-y/--yesonnpx.
npx skills add -y a-tokyo/agent-skills --skill production-grade --skill tribunal --skill database-documentation
npx skills add -y github/awesome-copilot --skill autoresearch
Summary
Folds the transferable lessons from our org's shared ESLint-config release (and its staged consumer migration) into the skill, plus one production regression fix its confirmation run caught.
eslint --print-configwhen text-grep misses — validated on a scratch preset consumer (268 sonarjs rules, cc 15){}in templates + anti-defang list entryeslint-config-next@next/eslint-plugin-nextdirectly.d.tsno-var override ·no-alphabetical-sort/e2e/**relaxations ·prefer-read-only-propsoffnoFallthroughCasesInSwitchnoUncheckedIndexedAccessdocumented as deliberate omissionsonarjs/deprecationoffRegression fix (found by this release's confirmation run): v0.0.2's consent gate could stall non-interactive runs with an unanswerable approval question. Phase 0 now defines non-interactive (no reply channel) and forbids asking in it; the consent gate's non-interactive branch is the
TODO(skills-install)block. The benchmark prompt also regained its designed do-not-ask line.Also: all deferral/roadmap wording removed from shipped text (no-op context); version synced in frontmatter + marketplace.json.
Test plan
🤖 Generated with Claude Code