fix(skill-management): remove broken agent refs and widen audit recall - #570
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Test plan outputPre-commit hooks (lefthook) on commit 9231a76: |
📚 Documentation Check ✅Verdict: Passed Plugin version was bumped correctly (1.2.0 to 1.3.0) in plugin.json and mirrored in the root CLAUDE.md table. No new skills/agents/commands were added, so no other doc updates are required. PR #570 refines skill-management plugin content only: fixes dangling agent references in SKILL.md/README, drops stale PROACTIVELY wording, and tightens the analysis rubric and output-length guidance. The required version bump was made and is consistent between plugin.json and the root CLAUDE.md table. No new components were added, so Notion marketplace and plugin CLAUDE.md updates are not required. Changelog is auto-generated via nx release, not a manual file. Minor non-blocking note: given these are corrective fixes rather than new features, a patch bump would have matched the repo's own semver guidance slightly better than the minor bump used. Suggestions (1)
🤖 Generated by Claude Documentation Validator | Mode: |
|
● Reviewed · 2026-08-05 23:41 UTC · 3 reviews · view run ↗
Removes three dead subagent dispatch targets from AssessmentThe latest push settles the DECISION the description left open: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9231a76e03
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - **Trigger-quality.** Start from `weak_or_missing_description` and | ||
| `no_trigger_language`, and flag any other description that reads weak on the | ||
| rubric. Deep-read the **writable** items you'll actually propose changing. Read |
There was a problem hiding this comment.
When map/full mode tries to find weak descriptions that the script did not flag, the skill has only been told to read inventory.md; that file renders each entry as desc (truncated) with only the first ~16 words, so an unflagged description can look fine in the table while failing the rubric later in the full text. This makes the new coverage-first audit miss precisely the unflagged trigger-quality issues it now promises to catch; have this path read inventory.json or otherwise include full descriptions before judging unflagged rows.
Useful? React with 👍 / 👎.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Tip
💬 Commented — see full review in the sticky comment ↑
There was a problem hiding this comment.
Note
✅ Approved — see full review in the sticky comment ↑
5576008 to
4b3ce0f
Compare
github-actions
Bot
changed the title
…grate-config-to-opus-5 Remove three phantom agent references, make the trigger-quality subagent dispatch conditional, make the analysis pass coverage-first, and add deliverable-length lines to both output sections. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
analysis-rubric.md opened with "don't invent problems" and nothing on the other side. The intent is right, but in an auditing prompt an unbalanced precision instruction reads to Opus 5 as license to drop anything it is uncertain about -- which is a recall filter applied at discovery time. Added the coverage counterweight: the guard constrains what you assert, not what you look at. Report every issue found, mark the uncertainty, and let the reader filter. A silently omitted real finding fails the same way a fabricated one does, and is harder to notice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
audit-patterns.md lists "PROACTIVELY in agent descriptions" as a Delta 2 smell to search for, and skill-doctor's own description used it. Shipping the pattern the plugin tells users to flag undercuts the guidance. The trigger itself is kept -- skill-doctor still applies after a repeatable multi-step workflow, especially right after a PR. Only the nudge word is gone. The PR-time prompt is delivered by hooks/pr-skill-doctor-prompt.cjs, not by this description, so nothing is lost by removing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
4b3ce0f to
48a0f0f
Compare
1 participant
Applies the Opus 5 migration audit to
packages/plugins/skill-management/. Every.mdunder the plugin was read in full; only files with an actual migration defect were touched.Deltas addressed
Broken references (dispatch names verified against disk)
skills/skill-doctor/SKILL.mdrouted deep work to four names, three of which do not exist:agent-capability-analystname:frontmatter anywhere in the marketplace or~/.claudeclaude-agent-discoveryagent-optimizerprompt-engineerprompt-engineer-agent(development-productivity)development-productivity:prompt-engineer-agentVerification — the only hits anywhere on disk for the three phantom names are copies of this same plugin:
Also corrected:
update-claude-mdwas described as a skill; it is a command (development-productivity/commands/update-claude-md.md). Now referenced as/update-claude-md.The same
prompt-engineercorrection was applied inREADME.md.Delta 2 — delegation
skill-doctorStep 3 mandateddeep-read them **via a subagent**for what its own sentence called "the handful of writable items you'll actually propose changing". Now conditional: read directly when there are only a few, delegate to a single subagent only when the set is large enough that inline reading would crowd out main context. An explicit when-NOT-to-delegate clause is stated inline ("a handful of reads is cheaper inline than a dispatch").Delta 3 — recall filter at discovery time
skill-doctorStep 3 told the analysis pass toUse the duplicate_names / near_duplicate_descriptions / weak_or_missing_description / no_trigger_language flags, which reads as a discovery-time boundary: an overlap the script's heuristics did not flag gets silently dropped. Reshaped coverage-first — the flags are now a starting point, the whole map gets read, ranking happens at the end rather than during discovery. Bullet verbs changed from "Use" to "Start from".No emphasis stripping was needed.
CRITICAL|MUST|NEVER|ALWAYS|ABSOLUTEcounts across the plugin: 2 inaudit-patterns.md, 1 inmigrate-config-to-opus-5/SKILL.md, 0 everywhere else — all load-bearing.Delta 4 — output length
skills/migrate-config-to-opus-5/SKILL.md## Outputhad no deliverable-length line, while its ownreferences/audit-patterns.md:54is the text instructing other configs to add one. The skill now follows its own advice.skill-doctorStep 5's consolidated menu now caps each entry at a line or two, with detail deferred to the items the user actually picks.Verified, deliberately unchanged
migrate-configStep 5'sclaude -pprobe and Step 2's "spot-check each proposed edit against the actual file" both check work against external reality, which is grounding, not self-review.skill-doctor's "suggest re-running inventory.py to confirm the flag cleared" is likewise a tool re-run, not a re-read of just-written output.audit-patterns.md— re-verified and correct, left untouched: Opus 5 $5/$25 vs Sonnet 5 $3/$15 → ~1.7x (with the ~2.5x intro-pricing caveat through 2026-08-31), Haiku $1/$5 → ~5x under Opus; model IDsclaude-opus-5/claude-sonnet-5/claude-haiku-4-5/claude-fable-5with the alias preference;budget_tokens400s on the Claude 5 family.count CRITICAL|MUST|NEVER|ALWAYS per file) is arithmetic over real grep output.MultiEdit/Task(subagent_type:)— no occurrences in this plugin.memory-doctor— referenced but does not exist on disk. Left as-is because the reference is explicitly hedged ("thememory-doctorskill if installed"), so it cannot fail at dispatch.DECISION (not resolved here — needs a call)
references/audit-patterns.md:24lists"PROACTIVELY" in agent descriptionsas a Delta 2 smell to flag. Its siblingskills/skill-doctor/SKILL.md:11says "Also use PROACTIVELY after finishing a multi-step workflow that the user is likely to repeat".hooks/pr-skill-doctor-prompt.cjsexists to reinforce.Either resolution is defensible; picking one is a judgment call, so nothing was changed.
Version
.claude-plugin/plugin.json1.2.0 → 1.3.0 (MINOR: dispatch targets removed, delegation and discovery behavior changed), and the matching row in the rootCLAUDE.mdtable. plugin.json and the root table agreed at 1.2.0 before this change — no drift.Test plan
Output pasted in a follow-up comment. Pre-commit (lefthook) ran clean on the commit:
🤖 Generated with Claude Code
AI-Generated Description
Summary
Applies the Opus 5 migration audit (from the
migrate-config-to-opus-5skill added in #562) topackages/plugins/skill-management/itself. Every.mdunder the plugin was read in full; only fileswith an actual defect were touched. 6 files, +42/-22, docs only — no code, hooks, or CI.
Three commits, four defect classes: broken dispatch targets, recall filters applied at discovery time,
unconditional delegation, and missing output-length instructions.
1. Broken dispatch references
skills/skill-doctor/SKILL.mdrouted deep work to four agent names. Three do not exist, so thosebullets could only ever fail at dispatch:
agent-capability-analystclaude-agent-discoveryagent-optimizerprompt-engineerprompt-engineer-agentdevelopment-productivity:prompt-engineer-agentThe filename is
prompt-engineer.mdbut the dispatch name in frontmatter isprompt-engineer-agent—the path was the likely source of the original error. The surviving reference now carries an explicit
"verify the name resolves before dispatching; if that plugin isn't installed, do the tuning inline"
fallback, so a missing plugin degrades instead of failing.
Also corrected:
update-claude-mdwas described as a skill. It is a command(
packages/plugins/development-productivity/commands/update-claude-md.md), now referenced as/update-claude-md. The sameprompt-engineercorrection was applied inREADME.md.2. Recall filters applied at discovery time (two sites)
skill-doctor/SKILL.mdStep 3 told the analysis pass toUse the duplicate_names / near_duplicate_descriptions / weak_or_missing_description / no_trigger_language flags. Read as adiscovery-time boundary, an overlap the script's keyword/threshold heuristics did not flag gets
silently dropped. Reshaped coverage-first: the flags are a starting point, the whole map gets read,
ranking happens at the end rather than during discovery. Bullet verbs changed from "Use" to "Start
from".
skill-doctor/references/analysis-rubric.mdopened with "don't invent problems" and nothing on theother side. The intent is right, but an unbalanced precision instruction in an auditing prompt reads to
Opus 5 as license to drop anything it is uncertain about — the same recall filter, one level down. Added
the counterweight: the guard constrains what you assert, not what you look at; report the finding
and mark the uncertainty rather than dropping it. A silently omitted real issue fails the same way a
fabricated one does, and is harder to notice.
3. Delegation made conditional
skill-doctorStep 3 mandated deep-reading via a subagent for what its own sentence called "thehandful of writable items you'll actually propose changing" — a dispatch costing more than the reads it
replaces. Now conditional: read directly when there are only a few, delegate to a single subagent only
when the set is large enough that inline reading would crowd out main context. The when-NOT-to-delegate
case is stated inline ("a handful of reads is cheaper inline than a dispatch") rather than left implicit.
4. Output length
skills/migrate-config-to-opus-5/SKILL.md## Outputhad no deliverable-length line, while its ownreferences/audit-patterns.md:54is the text instructing other configs to add one. The skill nowfollows its own advice.
skill-doctorStep 5's consolidated menu now caps each entry at a line or two, with detail deferredto the items the user actually picks.
5. PROACTIVELY self-inconsistency (resolved in
48a0f0f)references/audit-patterns.md:24lists"PROACTIVELY" in agent descriptionsas a Delta 2 smell toflag, and
skill-doctor's own description used it. Shipping the pattern the plugin tells users to flagundercuts the guidance, so the word is gone.
The trigger is kept — skill-doctor still applies after a repeatable multi-step workflow, especially
right after opening a PR. Only the nudge word was removed, and the PR-time prompt is delivered by
hooks/pr-skill-doctor-prompt.cjs, not by this description, so no behavior is lost. The plugin's onlyremaining
PROACTIVELYis now theaudit-patterns.mdline that defines the smell.Verified, deliberately unchanged
migrate-configStep 5'sclaude -pprobe and Step 2's"spot-check each proposed edit against the actual file" both check work against external reality,
which is grounding, not self-review. skill-doctor's "re-run
inventory.pyto confirm the flagcleared" is a tool re-run, not a re-read of just-written output.
CRITICAL|MUST|NEVER|ALWAYS|ABSOLUTEappears on 2 lines inaudit-patterns.md, 1 inmigrate-config-to-opus-5/SKILL.md, and 1 inanalysis-rubric.md— thelast being
heavy-handed ALL-CAPS MUSTs, prose about emphasis rather than emphasis itself. Allload-bearing.
audit-patterns.md— re-verified correct: Opus 5 $5/$25 vs Sonnet 5 $3/$15 →~1.7x (with the intro-pricing caveat through 2026-08-31), Haiku $1/$5 → ~5x under Opus; model IDs and
the alias preference;
budget_tokenson the Claude 5 family.arithmetic over real grep output.
MultiEdit/Task(subagent_type:)— no occurrences in this plugin.memory-doctor— referenced but not on disk. Left as-is: the reference is explicitly hedged("the
memory-doctorskill if installed"), so unlike the three names above it cannot fail atdispatch.
Out of scope — pre-existing broken refs elsewhere
Two of the phantom names are also referenced outside this plugin. They are equally broken, but they
belong to another plugin and to
docs/, so they would need their own version bump and are nottouched here:
Worth a follow-up. Note that
spec-workflow/skills/implement-spec/SKILL.mdno longer appears in thissweep — that reference was already removed by #567.
Version
.claude-plugin/plugin.json1.2.0 → 1.3.0 (minor: dispatch targets removed, delegation anddiscovery behavior changed), with the matching row in the root
CLAUDE.mdtable updated in the samecommit. The two agreed at 1.2.0 beforehand — no pre-existing drift.
Test plan
node scripts/validate-plugin.cjs packages/plugins/skill-management→ Validation PASSED(
skill-management v1.3.0, 2 skills, 3 commands, 2 hooks)the greps above
PROACTIVELYsweep across the plugin returns only theaudit-patterns.mdline that defines thesmell
bunx markdownlint-cli2 --fix→ 0 issues;bunx nx format:write --uncommittedappliedupdate-lockfile)