Skip to content

app-ai-guardrails v0.0.3: org-preset lint source, greenfield-ideal thresholds, eslint lessons - #17

Merged
a-tokyo merged 3 commits into
mainfrom
feature/app-ai-guardrails-0.0.3
Jul 20, 2026
Merged

app-ai-guardrails v0.0.3: org-preset lint source, greenfield-ideal thresholds, eslint lessons#17
a-tokyo merged 3 commits into
mainfrom
feature/app-ai-guardrails-0.0.3

Conversation

@a-tokyo

@a-tokyo a-tokyo commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Summary

Folds the transferable lessons from our org's shared ESLint-config release (and its staged consumer migration) into the skill, plus one production regression fix its confirmation run caught.

Lesson Disposition
Org preset package (factory + thin overrides) Adopted as the preferred Phase-0 lint source when one exists; inline canon remains the self-contained default. Scorer resolves via eslint --print-config when text-grep misses — validated on a scratch preset consumer (268 sonarjs rules, cc 15)
Flat-config option retention (severity-only override keeps old options) Adopted: explicit {} in templates + anti-defang list entry
ESLint ≥10 breaks eslint-config-next Adopted as the documented escape route: @next/eslint-plugin-next directly
.d.ts no-var override · no-alphabetical-sort/e2e/** relaxations · prefer-read-only-props off Adopted with rationale comments
tsconfig noFallthroughCasesInSwitch Adopted; noUncheckedIndexedAccess documented as deliberate omission
Coverage floors Raised to the greenfield ideal — 90, branches 85 (old 85/71/76/86 were one repo's brownfield ratchet floors). Scorer is behavioral, ledger unaffected; golden regenerated, self-test ALL CHECKS PASSED
antfu-foundation wholesale switch Rejected: the canon layers onto what the scaffolder generates
sonarjs/deprecation off Rejected (kept ON): a guardrails scaffold wants the deprecation signal

Regression fix (found by this release's confirmation run): v0.0.2's consent gate could stall non-interactive runs with an unanswerable approval question. Phase 0 now defines non-interactive (no reply channel) and forbids asking in it; the consent gate's non-interactive branch is the TODO(skills-install) block. The benchmark prompt also regained its designed do-not-ask line.

Also: all deferral/roadmap wording removed from shipped text (no-op context); version synced in frontmatter + marketplace.json.

Test plan

  • Self-test (fixtures regenerated at new floors): ALL CHECKS PASSED (golden=100 / bare=22)
  • Preset-path probe (scratch consumer of a real preset package): maximal-ruleset + complexity resolve correctly
  • Behavioral confirmation on the final text: fresh sonnet one-shot run → 94, all gates green, no question-stall
  • Compliance: SKILL.md 237/500 · description 949/1024 · zero defer/roadmap greps · all scripts syntax-clean

🤖 Generated with Claude Code

…resholds, eslint lessons, non-interactive fix

Learnings folded from an org shared-config release and its staged consumer migration:
- Phase 0 'lint source' parameter: org preset package preferred when one exists (thin consumer:
  factory + overrides only); inline canon stays the self-contained default. Scorer resolves rules
  via eslint --print-config when text-grep misses, so preset-composed repos score identically
  (validated: scratch consumer resolved 268 sonarjs rules, cognitive-complexity 15).
- Coverage thresholds raised to the greenfield ideal: 90 everywhere, branches 85 (per-stack
  mappings in canon/coverage.md). The inherited 85/71/76/86 were one repo's brownfield ratchet
  floors. Scorer checks enforcement behaviorally, so the published ledger is unaffected; golden
  regenerated, self-test ALL CHECKS PASSED.
- ESLint flat-config option-retention gotcha (explicit {} on overridden rules) + anti-defang entry;
  ESLint>=10 escape route (@next/eslint-plugin-next direct); .d.ts no-var override;
  no-alphabetical-sort + e2e/** test relaxations; prefer-read-only-props off with rationale;
  tsconfig noFallthroughCasesInSwitch (+ documented noUncheckedIndexedAccess omission).
- Non-interactive question-stall fix (v0.0.2 regression, caught by this release's confirmation
  run): Phase 0 defines non-interactive and forbids asking when no reply can arrive; consent
  gate's non-interactive branch is the TODO(skills-install) block. Benchmark prompt regained its
  designed do-not-ask line. Confirmation: fresh sonnet run scored 94, all gates green.
- Shipped-text deferral/roadmap wording removed (no-op context per writing-great-skills).

Version 0.0.3 in frontmatter + marketplace.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the app-ai-guardrails skill and its benchmark/scorer for the v0.0.3 release, incorporating lessons from an org-shared ESLint preset migration, raising “greenfield ideal” coverage floors, and hardening non-interactive behavior to avoid stalling runs.

Changes:

  • Bumped app-ai-guardrails to v0.0.3 and synced the version in .claude-plugin/marketplace.json.
  • Raised coverage thresholds across canon docs, adapters, and benchmark fixtures (generally to 90%+ with stack-appropriate axes).
  • Improved lint/scoring guidance and benchmark behavior: org-preset lint source path, ESLint flat-config option-retention note, “do not ask questions” benchmark prompt, and scorer support for preset-composed ESLint configs via eslint --print-config.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
skills/app-ai-guardrails/SKILL.md v0.0.3 bump; Phase 0 parameter/non-interactive wording updates and lint-source parameterization.
skills/app-ai-guardrails/references/canon/gate-interface.md Adds an ESLint flat-config anti-pattern entry about severity-only overrides retaining prior options.
skills/app-ai-guardrails/references/canon/coverage.md Updates per-stack coverage threshold table to the raised “greenfield ideal” floors.
skills/app-ai-guardrails/references/canon/agent-surface.md Clarifies consent gate behavior for non-interactive runs; adjusts TODO block wording.
skills/app-ai-guardrails/references/adapters/springboot.md Raises JaCoCo verification minimums and updates explanatory text.
skills/app-ai-guardrails/references/adapters/rust.md Raises cargo-llvm-cov fail-under thresholds.
skills/app-ai-guardrails/references/adapters/next.md Adds org-preset guidance, ESLint ≥10 escape route notes, test/e2e relaxations, d.ts overrides, and raised coverage thresholds.
skills/app-ai-guardrails/references/adapters/nest.md Raises coverage thresholds and points to reusing updated Next ESLint/org-preset guidance.
skills/app-ai-guardrails/references/adapters/go.md Raises Go coverage gate threshold; refines gomod-age note.
skills/app-ai-guardrails/references/adapters/django.md Raises pytest-cov fail-under threshold in narrative and config block.
skills/app-ai-guardrails/README.md Updates scope wording to “out of scope” (no roadmap language).
README.md Updates Claude Code plugin install examples to use the renamed marketplace (@agent-skills).
benchmarks/app-ai-guardrails/task/prompt-template.md Restores/strengthens “Do not ask questions” instruction to prevent stall regressions.
benchmarks/app-ai-guardrails/results/runs.csv Adds additional benchmark run records for the release confirmation.
benchmarks/app-ai-guardrails/fixtures/make-golden.sh Updates vitest coverage thresholds in golden fixture generation.
benchmarks/app-ai-guardrails/check-guardrails.mjs Enhances Next static-analysis probes to resolve rules via eslint --print-config when using preset-composed configs.
.claude-plugin/marketplace.json Renames marketplace to agent-skills and syncs app-ai-guardrails plugin version/description.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread skills/app-ai-guardrails/SKILL.md Outdated
Comment thread skills/app-ai-guardrails/references/canon/agent-surface.md
Comment thread skills/app-ai-guardrails/references/adapters/next.md Outdated
- SKILL.md: non-interactive rule now states stack precedence explicitly - stated/derivable ->
  proceed with defaults; missing -> abort (the one never-defaulted parameter); ask only when
  reachable. Resolves the proceed-vs-ask contradiction.
- agent-surface.md: TODO(skills-install) commands carry -y, consistent with the install-set rule
- next.md: suppression flags kept concrete but anchored to the currency rule (verify via
  eslint --help at use time)
- run-arm.sh: hard TERM/KILL watchdog replaces SIGALRM (observed runs hanging 60-99min past the
  soft alarm; CLI survives SIGALRM)

Behavioral note: run 14 (94, gates green) confirmed the exercised non-interactive path on this
text; the edits refine only the stack-missing abort edge, which the benchmark prompt cannot
exercise (it always states the stack).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Comment thread benchmarks/app-ai-guardrails/check-guardrails.mjs
Comment thread benchmarks/app-ai-guardrails/run-arm.sh Outdated
…ires a real file

- run-arm.sh: the KILL escalation now verifies the PID still runs a claude/node command
  (ps comm check) - a fast TERM exit within the 30s grace can no longer kill a reused PID
- check-guardrails.mjs: the --print-config fallback resolves against a real source file via the
  mutation-target walker instead of a fixed src/ path; no file -> keep text-grep verdicts.
  Fixes false negatives on non-src layouts. Self-test ALL CHECKS PASSED.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@a-tokyo

a-tokyo commented Jul 20, 2026

Copy link
Copy Markdown
Owner Author

Review dispositions (all rounds — surfaced here because inline replies collapse under outdated threads)

Round 1 (fixed in e361d9c):

  1. Non-interactive proceed-vs-ask contradiction — precedence now explicit: stack stated/derivable → proceed with declared defaults; stack missing in a no-reply session → abort (the one never-defaulted parameter); ask only when the user is reachable. Behavioral note: run 14 (94, all gates green) confirmed the exercised path; the edit sharpens the stack-missing abort edge the benchmark prompt cannot reach.
  2. TODO(skills-install) commands now carry -y — consistent with the install-set rule.
  3. Suppression flags — names kept (verified-real ESLint 10 core, production-used; concrete default > vague gesture) and anchored to the currency rule: verify against eslint --help at use time.

Round 2 (fixed in 2f77ec3):
4. ✅ Watchdog PID-reuse race — KILL escalation verifies the PID still runs a claude/node command before firing.
5. ✅ --print-config probe on non-src layouts — resolves a real source file via the teeth-probe walker; skips resolution (keeping text-grep verdicts) when none exists. Self-test ALL CHECKS PASSED.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

skills/app-ai-guardrails/references/canon/agent-surface.md:108

  • In the TODO(skills-install) block, -y is currently placed after add, which passes it to the skills CLI rather than to npx. That can still allow npx to prompt for installing the skills package (defeating the non-interactive fallback) and it’s inconsistent with the earlier guidance to run installs with -y/--yes on npx.
    npx skills add -y a-tokyo/agent-skills --skill production-grade --skill tribunal --skill database-documentation
    npx skills add -y github/awesome-copilot --skill autoresearch

@a-tokyo
a-tokyo merged commit 18ec0b0 into main Jul 20, 2026
1 check passed
@a-tokyo
a-tokyo deleted the feature/app-ai-guardrails-0.0.3 branch July 20, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants