Bring bundled skill frontmatter into Agent Skills spec compliance#56
Merged
Conversation
jbragg
force-pushed
the
fix/skill-frontmatter-spec-compliance
branch
5 times, most recently
from
May 11, 2026 16:57
ac90901 to
857b592
Compare
Every bundled SKILL.md in skills/* violated the Agent Skills spec (https://agentskills.io/specification) in two ways: - 'name': spec requires lowercase letters/numbers/hyphens matching the parent directory name. Existing skills used display strings. - 'allowed-tools': spec requires a space-separated string. Existing skills used YAML list syntax. Inspect_ai's read_skills() (called transitively by any inspect_swe agent via install_skills(), and any consumer using the skills-ref validator) rejects the existing format. Claude Code's own parser is more permissive and accepts both forms. Adds scripts/validate-skills.py and a 'make check-skills' target, wired into the CI workflow, so regressions to either field get caught at PR time.
jbragg
force-pushed
the
fix/skill-frontmatter-spec-compliance
branch
from
May 11, 2026 20:43
857b592 to
6fcf83a
Compare
rodneykinney
approved these changes
May 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every bundled SKILL.md in
plugins/asta/skills/*andplugins/asta-preview/skills/*violates the Agent Skills specification in two ways:namefield: spec requires "lowercase letters, numbers, and hyphens only" and "must match the parent directory name". Existing skills use display-style names (Semantic Scholar Lookup,Asta Library, etc.) that fail both rules.allowed-toolsfield: spec requires "a space-separated string"; existing skills use YAML list syntax.This PR rewrites the frontmatter to comply with the spec — no semantic content changes, just formatting. Display names move into the markdown body where they belong (they already appear as the first
# Headingon most skills).inspect_ai's
read_skills()(called transitively by inspect_swe agents viainstall_skills(), and any other consumer using the skills-ref validator) enforces the spec and rejects the existing files. Consumers that don't useread_skills(e.g. raw claude_code SKILL.md parser) accept the existing format, which is why this hasn't been noticed.Normalized via a one-off pyyaml-based script:
name:→ parent directory's kebab-case slug;allowed-tools: [list]→allowed-tools: <space-joined string>. Verified all 19 normalized SKILL.md files passinspect_ai.tool.read_skillsvalidation.Needed by: agent-baselines inspect_swe baseline solver, which clones asta-plugins host-side and passes skill dirs through inspect_swe's standard
skills=plumbing (which callsread_skillsand currently rejects every bundled skill).