auditor: match repo-root SKILL.md in the discover probe; seed video-shotcraft - #790
Open
xiaolai wants to merge 1 commit into
Open
auditor: match repo-root SKILL.md in the discover probe; seed video-shotcraft#790xiaolai wants to merge 1 commit into
xiaolai wants to merge 1 commit into
Conversation
…hotcraft The artifact probe had no pattern anchored at a repo-root SKILL.md — the canonical single-skill layout in the open Agent Skills spec, where the repo itself is the skill directory. `^[^/]+/SKILL\.md$` needs one leading directory and the `skills/`-prefixed patterns need that literal segment, so a root SKILL.md matched nothing. Found auditing Vincentwei1021/video-shotcraft (3.7k stars, 660 files, ~116 NL artifacts): it probed at 1 artifact — `.claude-plugin/plugin.json` alone — against a MIN_ARTIFACTS=5 floor, so discovery could never surface it. bin/nlpm-check finds root SKILL.md fine (rglob); the gap was probe-specific. The fix takes that repo 1 -> 2, still under the floor. Its real NL surface is a 104-card references/shots/ tree that no probe pattern reaches, so the repo is seeded by hand here (audit issue #789) rather than waiting on a broader change to how the probe counts reference layers.
xiaolai
force-pushed
the
auditor/seed-video-shotcraft
branch
from
August 7, 2026 04:34
ca52b4b to
eb5a023
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two changes, one cause:
Vincentwei1021/video-shotcraftis a 3.7k-star skill repo the pipeline could never have found.1. Discover probe missed repo-root
SKILL.mdThe artifact-count expression in
auditor-discover.ymlhad no pattern anchored at a repo-rootSKILL.md— the canonical single-skill layout in the open Agent Skills spec, where the repo itself is the skill directory:^[^/]+/SKILL\.md$requires one leading directoryskills/.*/SKILL\.md$and the.claude/skills/...variant require that literal segmentSo a bare root
SKILL.mdmatched nothing.bin/nlpm-checkhandles this layout fine (it rglobs forSKILL.md), which is why the gap went unnoticed — it is probe-specific.Blast radius, measured: video-shotcraft has 660 files and ~116 NL artifacts. The probe scored it 1 (
.claude-plugin/plugin.jsonalone) againstMIN_ARTIFACTS=5. Invisible to discovery, and absent from all 280 registry entries.Verified against the real tree: 1 → 2 after the fix.
2. Seeded the repo by hand
2 is still under the floor of 5, so the fix alone does not surface it. Seeded manually with
discovery_strategy: "manual-seed"and audit issue #789; the registry write went throughatomic-registry-write.shand the metadata counters were bumped by the same jq discover uses.Open question, not addressed here
The probe counts layout markers, not the reference layer. video-shotcraft's actual NL surface is 104 shot cards under
references/shots/plus 7 docs underreferences/, and no pattern reaches any of them — hence 2 rather than ~116. nlpm's own skills usereferences/the same way, so any repo shaped "one root skill + a large reference tree" is undercounted by roughly two orders of magnitude.Whether the probe should count a skill's
references/**/*.mdis a real change to discovery breadth, so I left it alone rather than deciding it inside a bug fix.Verification
python3 -m unittest discover -s testsbin/nlpm-check .Context
Manual audit of the repo at
0022ec4:nlpm-checkclean, security PASS, NL score 98 (SKILL.md 95, plugin.json 100,agents/openai.yaml100). One finding, contributed upstream as Vincentwei1021/video-shotcraft#28. Expected to clear the exemplar gate once the pipeline audits it.