From 6a0314c8e42856f39c86b5bdf32b2ee6b7e505de Mon Sep 17 00:00:00 2001 From: Yogesh Rao Date: Fri, 29 May 2026 13:35:42 +0530 Subject: [PATCH 1/2] feat: improve skill score for generate-agent-skills MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hey @srbouffard ๐Ÿ‘‹ I ran your skills through `tessl skill review` at work and found some targeted improvements. Here's the full before/after: | Skill | Before | After | Change | |-------|--------|-------|--------| | documentation-build | 94% | 94% | โ€” | | documentation-diataxis | 85% | 85% | โ€” | | documentation-review | 88% | 88% | โ€” | | documentation-structure | 79% | 79% | โ€” | | documentation-style | 94% | 94% | โ€” | | documentation-verify | 94% | 94% | โ€” | | **generate-agent-skills** | **65%** | **87%** | **+22%** | | generate-agent | 82% | 82% | โ€” | | generate-path-instructions | 88% | 88% | โ€” | | generate-prompt | 78% | 78% | โ€” | | generate-repo-instructions | 81% | 81% | โ€” | | migrate-harness-tests-to-state-transition-test | 75% | 75% | โ€” | | retrospective-artifacts | 88% | 88% | โ€” | | landscape-jira | 79% | 79% | โ€” |
Changes made to generate-agent-skills - **Rewrote description** to include concrete actions (creates/updates SKILL.md files, scaffolds directories, validates structure) and an explicit "Use when..." clause with natural trigger terms (`new agent skill`, `skill.md`, `skill definitions`, `skill templates`) - **Removed verbose emoji warning blocks** (๐Ÿšจ, โš ๏ธ, ๐Ÿ›‘) and replaced with concise mandatory constraints โ€” same rules, 75% fewer tokens - **Condensed Step 2** from 40+ lines to ~5 lines by removing inline resource type taxonomy (already covered in `references/BEST_PRACTICES.md`) and pointing there instead - **Simplified Step 3** scaffolding section โ€” removed redundant verification checkpoint and stop conditions, kept the essential command and naming regex - **Tightened Step 4** content generation โ€” removed explanations that duplicate reference files, kept frontmatter guidance and structure pattern references - **Consolidated Step 5** validation โ€” replaced 25-line checklist with concise confirmation criteria - **Streamlined Step 6** testing โ€” replaced verbose problem/solution pairs with compact iteration checklist - **Added Reference Index** table at bottom for quick navigation to all 5 reference files Net result: 357 โ†’ 138 lines (61% reduction) while preserving all workflow steps, validation checkpoints, and reference file pointers.
I also stress-tested your `documentation-style` skill against a few real-world task evals and it held up really well on MyST/reST syntax validation with cross-referenced style guide citations. Kudos for that. Honest disclosure โ€” I work at @tesslio where we build tooling around skills like these. Not a pitch โ€” just saw room for improvement and wanted to contribute. Want to self-improve your skills? Just point your agent (Claude Code, Codex, etc.) at [this Tessl guide](https://docs.tessl.io/evaluate/optimize-a-skill-using-best-practices) and ask it to optimize your skill. Ping me โ€” [@yogesh-tessl](https://github.com/yogesh-tessl) โ€” if you hit any snags. Thanks in advance ๐Ÿ™ --- skills/generate-agent-skills/SKILL.md | 301 ++++---------------------- 1 file changed, 41 insertions(+), 260 deletions(-) diff --git a/skills/generate-agent-skills/SKILL.md b/skills/generate-agent-skills/SKILL.md index fe0832a..86520c8 100644 --- a/skills/generate-agent-skills/SKILL.md +++ b/skills/generate-agent-skills/SKILL.md @@ -1,6 +1,6 @@ --- name: generate-agent-skills -description: Architects, generates, and validates Agent Skills. Enforces specification and best practices. Used any time an agent skill must be created or updated. +description: "Creates and updates SKILL.md files with YAML frontmatter, workflow steps, and bundle resources. Scaffolds directories via scaffold_skill.py and validates via validate_skill.py. Use when creating a new agent skill, writing a skill.md, updating skill definitions, or generating skill templates." compatibility: python3 allowed-tools: python3 ls grep cat mkdir metadata: @@ -10,27 +10,13 @@ metadata: # Agent Skill Architect Workflow -This skill guides you through creating high-quality Agent Skills following a proven 6-step process. +Create high-quality Agent Skills following a 6-step process: understand โ†’ plan โ†’ scaffold โ†’ write โ†’ validate โ†’ test. ---- - -## ๐Ÿšจ **CRITICAL WORKFLOW REQUIREMENTS** ๐Ÿšจ - -**Before you begin, understand these NON-NEGOTIABLE rules:** - -1. **You MUST run `scripts/scaffold_skill.py` in Step 3.** - Manual file creation is PROHIBITED. The scaffolding script ensures consistency. - -2. **You MUST use the generated templates.** - After scaffolding, templates exist in `references/`. Use them as your foundation. - -3. **You MUST run `scripts/validate_skill.py` in Step 5.** - Validation catches errors before they propagate. - -4. **You MUST follow all 6 steps in order.** - Skipping steps leads to non-compliant or broken skills. - -**If you bypass scaffolding scripts, you have FAILED this workflow.** +**Mandatory constraints:** +1. Run `scripts/scaffold_skill.py` in Step 3 โ€” manual file creation is prohibited. +2. Use the generated templates from `references/` as your foundation. +3. Run `scripts/validate_skill.py` in Step 5 before finalizing. +4. Follow all 6 steps in order. --- @@ -62,157 +48,47 @@ If working with an existing skill, analyze: ## Step 2: Planning Reusable Contents -Analyze the concrete examples from Step 1 to identify what **reusable resources** would help. - -### โš ๏ธ Critical Decision: Script vs. Checklist - -**Before planning scripts, ask:** "Is this task primarily analysis or computation?" - -**Analysis tasks** (reading, synthesizing, pattern recognition): -โ†’ Use **checklists** or **reference docs** for LLM to follow -โ†’ Examples: Repository analysis, code review, documentation synthesis - -**Computation tasks** (math, APIs, precise transformations): -โ†’ Use **scripts** for deterministic execution -โ†’ Examples: Schema validation, API calls, file format conversion - -**Real example from this session:** -- โŒ Initially planned `analyze_repo.py` script -- โœ… Corrected to `analysis_checklist.md` reference -- **Why:** Repository analysis = LLM strength (reading, pattern detection, synthesis) - -**See `references/BEST_PRACTICES.md` ยง6 for detailed decision flowchart.** - ---- - -### Ask for each example: -1. How would I execute this task from scratch? -2. What scripts, references, or assets would make this repeatable? -3. **Is this analysis (LLM) or computation (script)?** - -### Resource Types: +Analyze examples from Step 1 to identify reusable resources. For each, decide: analysis tasks โ†’ `references/` (checklists, patterns, domain knowledge); computation tasks โ†’ `scripts/` (math, APIs, validation); output artifacts โ†’ `assets/` (templates, images, seed data). See `references/BEST_PRACTICES.md` ยง6 for the decision flowchart. -**scripts/** - For deterministic operations only: -- โœ… Math/computation (calculations, aggregations) -- โœ… External interactions (API calls, database queries) -- โœ… Precise transformations (file format conversion, schema validation) -- โœ… Repetitive generation (boilerplate rendering) -- โŒ Analysis tasks (use checklists instead) -- โŒ Pattern recognition (LLM excels at this) - -**references/** - For LLM-driven analysis and knowledge: -- โœ… Checklists for systematic analysis (e.g., repository discovery) -- โœ… Pattern libraries (e.g., positive constraint conversions) -- โœ… API documentation (endpoints, parameters) -- โœ… Domain knowledge (company policies, industry standards) -- โœ… Decision trees and workflows - -**assets/** - For files used in output: -- โœ… Templates (documents, slides, boilerplate code) -- โœ… Images (logos, icons, diagrams) -- โœ… Fonts (typography files) -- โœ… Seed data (sample datasets, fixtures) - -**Output:** A list of specific files to create with correct categorization (script vs reference) +**Output:** A list of specific files to create with correct categorization. --- ## Step 3: Skill Scaffolding -**โš ๏ธ MANDATORY STEP - DO NOT SKIP โš ๏ธ** +Run the scaffolding script (use `--simple` for SKILL.md only): -You MUST execute the scaffolding script. Manual file creation is PROHIBITED. - -### Command: ```bash python3 scripts/scaffold_skill.py --name ``` -### Options: -- **Default mode:** Creates SKILL.md + example files in scripts/, references/, assets/ -- **Simple mode:** Use `--simple` flag for minimal structure (SKILL.md only) - -**The script will:** -- โœ… Validate naming conventions (lowercase, hyphens, alphanumeric) -- โœ… Create skill directory in .github/skills/ -- โœ… Generate SKILL.md with structuring guidance -- โœ… Create example files to demonstrate resource organization - -**Note:** The script auto-detects `.github/skills` from git root. Naming must match regex: `^[a-z0-9][a-z0-9-]*[a-z0-9]$` - ---- - -### โœ… **Verification Checkpoint** - -After running the scaffolding script, confirm these files exist: -```bash -ls -la .github/skills// -``` - -**Expected output:** -- `SKILL.md` (with "Structuring This Skill" guidance section) -- `scripts/example.py` (placeholder script) -- `references/example_reference.md` (placeholder reference) -- `assets/README.md` (if using default mode) - -**๐Ÿ›‘ STOP CONDITIONS:** -- If `SKILL.md` does NOT exist โ†’ Scaffolding failed, do NOT proceed -- If you created files manually โ†’ You have violated the workflow, DELETE and re-run script -- If the script reported errors โ†’ Fix errors before proceeding to Step 4 +Validates naming (`^[a-z0-9][a-z0-9-]*[a-z0-9]$`), creates the directory under `.github/skills/`, and generates SKILL.md with placeholders. Verify with `ls -la .github/skills//` before proceeding. --- ## Step 4: Content Generation -Populate the skill with actual content. - -### 4.1: Implement Reusable Resources First - -Start with scripts/, references/, and assets/ identified in Step 2. - -**For scripts:** -- Replace `scripts/example.py` with actual implementation -- Test by running: `python3 scripts/.py` -- Ensure error messages are descriptive (print to stderr) - -**For references:** -- Replace `references/example_reference.md` with actual docs -- Keep SKILL.md lean - move details here -- For large files (>100 lines), add Table of Contents +### 4.1: Implement Reusable Resources -**For assets:** -- Add actual template files, images, fonts -- Replace or delete `assets/README.md` -- Use descriptive filenames - -**Important:** Delete any example files you don't need! +Replace placeholder files from scaffolding with actual implementations: +- **scripts/**: Replace `example.py` with real scripts. Test each: `python3 scripts/.py` +- **references/**: Replace `example_reference.md` with actual docs. Keep SKILL.md lean โ€” move details here. +- **assets/**: Add template files, images, seed data. Delete unused placeholders. ### 4.2: Write SKILL.md Content -Follow the structuring guidance embedded in the generated SKILL.md template. - -**Choose your structure pattern:** -- **Workflow-Based:** Sequential processes (see `references/workflows.md`) -- **Task-Based:** Tool collections with different operations -- **Reference/Guidelines:** Standards, specifications, coding rules -- **Capabilities-Based:** Integrated systems with multiple features - -**Key elements:** +**Frontmatter (YAML):** +- `name`: Must match directory name exactly. +- `description`: High-entropy, keyword-rich, 3rd person. Include a "Use when..." clause with trigger scenarios and concrete capabilities. + - Example: `"Processes PDF documents for form filling, text extraction, and merging. Use when working with PDF files or when user requests document manipulation tasks."` -1. **Frontmatter (YAML):** - - `name`: Must match directory name exactly - - `description`: High-entropy, keyword-rich, 3rd person - - Include WHEN to use this skill (triggers) - - Include WHAT the skill does (capabilities) - - Example: "Processes PDF documents for form filling, text extraction, and merging. Use when working with PDF files or when user requests document manipulation tasks." +**Body (Markdown):** +- Use imperative form ("Run the script", not "You should run"). +- Reference scripts/references explicitly by path. +- Choose a structure pattern: workflow-based, task-based, reference/guidelines, or capabilities-based (see `references/workflows.md`). +- Consult `references/BEST_PRACTICES.md` for the Freedom Scale, `references/output-patterns.md` for output formatting. -2. **Body (Markdown):** - - Use imperative/infinitive form ("Run the script", not "You should run") - - Reference scripts/references explicitly by path - - Consult `references/BEST_PRACTICES.md` for the "Freedom Scale" - - Consult `references/output-patterns.md` for output formatting - -**Delete the "Structuring This Skill" section** when done - it's guidance only! +Delete the "Structuring This Skill" guidance section when done. ### 4.3: Design Patterns @@ -231,126 +107,31 @@ Follow the structuring guidance embedded in the generated SKILL.md template. ## Step 5: Validation -**โš ๏ธ MANDATORY STEP - DO NOT SKIP โš ๏ธ** - -Run the validation script to ensure specification compliance. +Run the validation script to ensure specification compliance: -### Command: ```bash -python3 scripts/validate_skill.py --path +python3 scripts/validate_skill.py --path .github/skills/ ``` -**Example:** -```bash -python3 scripts/validate_skill.py --path .github/skills/diagnose-ci-failure -``` - -### What it checks: -- โœ… Directory naming regex (`^[a-z0-9][a-z0-9-]*[a-z0-9]$`) -- โœ… SKILL.md exists -- โœ… YAML frontmatter has required fields (name, description) -- โœ… Name in YAML matches directory name -- โš ๏ธ Advisory: Presence of references/ and scripts/ (warnings only) - -**If validation fails:** -- Read the error output carefully -- Fix critical violations immediately -- Warnings are informational (acceptable for simple skills) +Checks: directory naming, SKILL.md exists, required frontmatter fields (`name`, `description`), name matches directory. Warnings about missing `references/` or `scripts/` are advisory. -**When valid:** Proceed to testing! - ---- - -### โœ… **Post-Validation Checklist** - -Before proceeding to Step 6, confirm: - -**Workflow Compliance:** -- [ ] I RAN `scripts/scaffold_skill.py` (Step 3) -- [ ] I USED the generated templates from scaffolding -- [ ] I CONSULTED `references/TEMPLATES.md` and `references/BEST_PRACTICES.md` (Step 4) -- [ ] I RAN `scripts/validate_skill.py` (Step 5) -- [ ] Validation script reported SUCCESS (no critical errors) - -**Content Quality:** -- [ ] YAML frontmatter includes `name` and `description` -- [ ] Description is high-entropy and keyword-rich -- [ ] No "Structuring This Skill" guidance section remains in SKILL.md -- [ ] Example files (`example.py`, `example_reference.md`) are deleted or replaced -- [ ] Scripts are in `scripts/`, references in `references/`, templates in `assets/` - -**๐Ÿ›‘ STOP CONDITION:** -If you did NOT run the scaffolding script or manually created files, STOP and re-do from Step 3. +Fix critical errors before proceeding. Confirm scaffolding script was used, frontmatter includes a "Use when..." clause, and no placeholder files remain. --- ## Step 6: Testing and Iteration -After creating the skill, test and refine based on real usage. - -### Testing Workflow: - -1. **Test with real examples** from Step 1 - - Does the skill trigger on expected queries? - - Do scripts execute without errors? - - Is output quality acceptable? - -2. **Identify friction points:** - - Are instructions clear enough? - - Are there missing scripts or references? - - Is context loaded efficiently? - -3. **Iterate on improvements:** - - Update SKILL.md for clarity - - Add missing examples or edge cases - - Optimize script error handling - - Split large references if needed (progressive disclosure) - -4. **Re-validate** after changes - -### Common Iteration Patterns: - -**Problem:** Skill isn't triggering when expected -**Solution:** Enhance description with more keywords and trigger scenarios - -**Problem:** Agent struggles with workflow steps -**Solution:** Add decision tree or flowchart; consult `references/workflows.md` - -**Problem:** Context feels bloated -**Solution:** Move content from SKILL.md to references/; add grep hints - -**Problem:** Scripts fail in edge cases -**Solution:** Add error handling; print descriptive messages to stderr - -**Problem:** Output quality inconsistent -**Solution:** Add templates or validation checklist; see `references/output-patterns.md` - -### When to Stop Iterating: - -โœ… Skill triggers reliably on target queries -โœ… Workflows execute without confusion -โœ… Output quality meets requirements -โœ… No critical errors in testing - ---- - -## Knowledge Retrieval - -If questions arise during skill creation: - -**Specification questions** (naming, structure, required files): -โ†’ Read `references/SPECIFICATION.md` - -**Best practices** (context economy, freedom scale, anti-patterns): -โ†’ Read `references/BEST_PRACTICES.md` - -**Templates and examples** (frontmatter, structure patterns): -โ†’ Read `references/TEMPLATES.md` +Test the skill with real examples from Step 1, then iterate: -**Workflow design** (sequential, conditional, iterative): -โ†’ Read `references/workflows.md` +1. **Trigger testing**: Does the skill activate on expected queries? If not, add keywords to the description. +2. **Execution testing**: Do scripts run without errors? Is output quality acceptable? +3. **Context efficiency**: If SKILL.md feels bloated, move content to `references/`. +4. **Re-validate** after each round of changes with `scripts/validate_skill.py`. -**Output formatting** (templates, examples, validation): -โ†’ Read `references/output-patterns.md` +## Reference Index -**Do not hallucinate answers.** Always consult the authoritative sources. +- **Specification** (naming, structure): `references/SPECIFICATION.md` +- **Best practices** (context economy, freedom scale): `references/BEST_PRACTICES.md` +- **Templates** (frontmatter, structure patterns): `references/TEMPLATES.md` +- **Workflows** (sequential, conditional, iterative): `references/workflows.md` +- **Output patterns** (templates, validation checklists): `references/output-patterns.md` From 95459fa09090f6852b3b7309cf2758c252484acf Mon Sep 17 00:00:00 2001 From: Yogesh Rao Date: Wed, 3 Jun 2026 15:07:31 +0530 Subject: [PATCH 2/2] feat: restore LLM behavioral anchors per review feedback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address review comments on PR #45 while keeping the token savings: - Step 2: restore the analyze_repo.py -> analysis_checklist.md reasoning anchor inline, pointing to BEST_PRACTICES.md ยง6 for the full flowchart - Step 3: restore explicit STOP CONDITIONS (halt on scaffold failure / manual file creation / script errors) - Step 5: restore a trimmed self-compliance checklist (ran scaffold script, no critical errors, "Use when..." clause, no placeholders) - Reference Index: restore "Do not hallucinate answers" instruction These act as behavioral anchors that halt the agent on failure rather than merely inform it. --- skills/generate-agent-skills/SKILL.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/skills/generate-agent-skills/SKILL.md b/skills/generate-agent-skills/SKILL.md index 86520c8..4cd7dcd 100644 --- a/skills/generate-agent-skills/SKILL.md +++ b/skills/generate-agent-skills/SKILL.md @@ -48,7 +48,9 @@ If working with an existing skill, analyze: ## Step 2: Planning Reusable Contents -Analyze examples from Step 1 to identify reusable resources. For each, decide: analysis tasks โ†’ `references/` (checklists, patterns, domain knowledge); computation tasks โ†’ `scripts/` (math, APIs, validation); output artifacts โ†’ `assets/` (templates, images, seed data). See `references/BEST_PRACTICES.md` ยง6 for the decision flowchart. +Analyze examples from Step 1 to identify reusable resources. For each, decide: analysis tasks โ†’ `references/` (checklists, patterns, domain knowledge); computation tasks โ†’ `scripts/` (math, APIs, validation); output artifacts โ†’ `assets/` (templates, images, seed data). + +**Real example:** A repository-analysis task was initially planned as an `analyze_repo.py` script, then corrected to an `analysis_checklist.md` reference โ€” repository analysis is an LLM strength (reading, pattern detection, synthesis), not deterministic computation. When in doubt, prefer a checklist over a script for analysis work. See `references/BEST_PRACTICES.md` ยง6 for the full decision flowchart. **Output:** A list of specific files to create with correct categorization. @@ -64,6 +66,11 @@ python3 scripts/scaffold_skill.py --name Validates naming (`^[a-z0-9][a-z0-9-]*[a-z0-9]$`), creates the directory under `.github/skills/`, and generates SKILL.md with placeholders. Verify with `ls -la .github/skills//` before proceeding. +**Stop conditions:** +- If `SKILL.md` does NOT exist after running the script โ†’ do NOT proceed; the scaffolding failed. +- If you created files manually instead of running the script โ†’ delete them and re-run the script. +- If the script reported errors โ†’ fix them before continuing to Step 4. + --- ## Step 4: Content Generation @@ -115,7 +122,13 @@ python3 scripts/validate_skill.py --path .github/skills/ Checks: directory naming, SKILL.md exists, required frontmatter fields (`name`, `description`), name matches directory. Warnings about missing `references/` or `scripts/` are advisory. -Fix critical errors before proceeding. Confirm scaffolding script was used, frontmatter includes a "Use when..." clause, and no placeholder files remain. +Fix critical errors before proceeding. Then confirm: +- [ ] Ran `scripts/scaffold_skill.py` (did not create files manually) +- [ ] Ran `scripts/validate_skill.py` with no critical errors +- [ ] Frontmatter `description` includes a "Use when..." clause +- [ ] No placeholder files (`example.py`, `example_reference.md`) remain + +If you did not run the scaffolding script or manually created files, STOP and re-do from Step 3. --- @@ -135,3 +148,5 @@ Test the skill with real examples from Step 1, then iterate: - **Templates** (frontmatter, structure patterns): `references/TEMPLATES.md` - **Workflows** (sequential, conditional, iterative): `references/workflows.md` - **Output patterns** (templates, validation checklists): `references/output-patterns.md` + +**Do not hallucinate answers.** Always consult the authoritative sources.