Skip to content

REPORT.md

Latest commit

 

History

History
255 lines (184 loc) · 15.8 KB

REPORT.md

File metadata and controls

255 lines (184 loc) · 15.8 KB
Raw

Blueprint: Architecting and Developing AI Agent Skills

This handbook serves as a comprehensive guide for designing, architecting, and implementing production-grade AI Agent Skills. Extracted from the architectural patterns of sophisticated existing skills, this report defines what a skill is, how to structure its repository, and the engineering principles required to build one that is reliable, scalable, and cross-platform.

1. Introduction

What is an AI Agent Skill?

An AI Agent Skill is a packaged set of instructions, prompt fragments, and (optionally) deterministic backend code designed to be consumed by an AI coding assistant (like Claude Code, Cursor, Aider, GitHub Copilot). It grants the host agent new capabilities, standardized workflows, or integration with external systems.

A skill essentially programs the behavior of an LLM-driven agent, acting as an orchestration layer that guides the agent through a complex, multi-step process.

Why Skills Exist

LLMs are highly capable but prone to hallucination, context drift, and inefficient tool usage when left unguided on complex tasks. Skills exist to:

  • Standardize Workflows: Ensure a multi-step task is executed the exact same way every time.
  • Save Tokens: Prevent the agent from reading unnecessary files or taking inefficient paths.
  • Combine Paradigms: Marry the deterministic reliability of traditional code (parsing, caching) with the fuzzy, semantic reasoning of LLMs.

Skill vs. Standalone AI App

Feature AI Agent Skill Standalone AI App
Runtime Runs inside the user's IDE or CLI agent. Runs in its own sandbox, container, or cloud.
Compute Borrows the user's local terminal and the host agent's LLM context. Brings its own LLM backend and execution environment.
Context Has immediate access to the user's open files and workspace. Must ingest context explicitly via API or upload.

When to Create a Skill

Build a skill when you have a repeatable workflow that requires a mix of deterministic actions (running CLI tools, reading files) and semantic decisions (summarizing, labeling, extracting intent), and you want this workflow accessible directly where developers work (the IDE/CLI).

2. High-Level Architecture

The lifecycle of an AI Agent Skill execution follows a strict pipeline to prevent the host agent from wandering off-script.

flowchart TD
    A[User Request / Slash Command] --> B[Trigger Match & Prompt Injection]
    B --> C[Context Collection & Validation]
    C --> D[Deterministic Execution via Bash/CLI]
    D --> E[Subagent Dispatch / LLM Operations]
    E --> F[Result Aggregation & Output]
    F --> G[Interactive Follow-up]
Loading
  1. User Request: The user types a command (e.g., /my-skill .).
  2. Trigger Match: The host framework identifies the skill and injects its SKILL.md (or equivalent orchestrator prompt) into the system prompt.
  3. Context Collection: The agent executes a deterministic command (e.g., a fast Python script) to map the workspace and return a minimal summary (avoiding token bloat).
  4. Execution Flow: The prompt guides the agent through strict steps (Step 1, Step 2, etc.), utilizing the host's Terminal/Bash tool to run backend code.
  5. Subagent Orchestration: For highly parallelizable semantic tasks, the skill instructs the host agent to dispatch multiple subagents.
  6. Result & Output: Artifacts are written to disk, and the agent summarizes the output to the user.

3. Repository Structure

A well-designed skill repository cleanly separates the LLM prompt instructions from the deterministic backend execution, and utilizes a build system to target multiple agent platforms.

my-skill-repo/
├── backend/                  # Deterministic code (Python, Go, TS)
│   ├── detect.py             # Fast file scanning/validation
│   ├── extract.py            # AST/RegEx extraction (zero-token tasks)
│   └── cli.py                # Entrypoint for Bash tool calls
├── fragments/                # Modular prompt components
│   ├── core/                 # The main orchestration logic
│   ├── dispatch/             # Platform-specific subagent dispatch rules
│   ├── references/           # Detailed sub-schemas loaded on demand
│   └── always-on/            # Context nudges for AGENTS.md / CLAUDE.md
├── tests/                    # Unit tests for the backend
├── build.py                  # "Skillgen" build script to compile prompts
└── pyproject.toml / package.json

Directory Responsibilities

  • backend/: Handles anything that doesn't require an LLM. Parsing, AST traversal, API fetching, caching, and file I/O. Never put prompt strings here unless they are part of a subagent payload.
  • fragments/: The raw material for your prompts. Separating prompts into fragments allows you to reuse sub-schemas and adapt the skill for different agent frameworks (e.g., frameworks that use an Agent tool vs. a Task tool).
  • always-on/: Contains small markdown snippets that the skill can inject into a project's global rules (.cursor/rules/, AGENTS.md) so the skill can subtly guide the agent even when not explicitly invoked.

4. Core Components

1. The Orchestrator Prompt (SKILL.md)

This is the "main()" function of your skill. It must be written defensively. It lists steps explicitly (Step 1, Step 2) and contains exact bash scripts the agent must run via its terminal tool. It dictates control flow (e.g., "If X is empty, stop and warn the user").

2. The Execution Engine

The local codebase that the Orchestrator Prompt invokes. The engine's job is to do heavy lifting and return dense, token-efficient JSON or summaries back to stdout, which the Orchestrator reads to decide the next step.

3. Always-On Context Nudges

A mechanism to persist behavior. A skill isn't just a one-off command; it often changes how the project operates. The skill should have an install routine that appends a rule to the workspace (e.g., "Always check output/report.json before answering architecture questions").

4. Subagent Specifications

When tasks need to scale (e.g., reading 100 files semantically), reading them sequentially in the main loop is too slow. The component architecture must include a prompt schema specifically designed to be passed to a disposable subagent.

5. Prompt Engineering Strategy

Prompt Modularity & The Build System

Do not hardcode one massive SKILL.md. Different AI coding assistants have different native tools (e.g., Claude Code uses Agent for subagents, Factory Droid uses Task, Cursor uses .mdc rules).

  • Strategy: Build a prompt compiler (e.g., skillgen).
  • Maintain a core.md with slots (e.g., @@DISPATCH_LOGIC@@).
  • Inject the correct instruction block at build time based on the target platform.

Variable Injection

Never ask the host agent to dynamically figure out complex file paths or environments.

  • Strategy: Write bash blocks in the prompt that resolve absolute paths before the agent acts on them. For example: 
    PROJECT_ROOT=$(pwd)
CHUNK_PATH="${PROJECT_ROOT}/output/.chunk_01.json"

Defensive Control Flow

LLMs tend to skip steps if they think they know the answer.

  • Strategy: Number your steps. Add explicit halts: "If this command returns an error, DO NOT proceed to Step 3. Tell the user what happened."

6. Execution Flow Pipeline

A robust skill executes in a predictable state machine.

Phase 1: Environment & Guardrails

  • Identify the correct interpreter (python3 vs .venv/bin/python).
  • Save state (e.g., .skill_python_env) to disk so subsequent bash blocks use the exact same environment without the LLM guessing.

Phase 2: Deterministic Fast-Pass

  • Execute backend scripts to map the domain (e.g., file counts, AST extraction).
  • Print a tiny summary to stdout for the LLM to read. (e.g., Code: 50 files. Docs: 10 files.)

Phase 3: The Parallel Subagent Dispatch

To achieve speed, the Orchestrator Prompt leverages the host's ability to spawn parallel tools.

  • The Trick: Instruct the agent to call the Agent or Task tool multiple times in the same response payload.
  • Each tool call contains a chunk of files and the Subagent Specification prompt.
  • Since LLMs execute parallel tool calls concurrently, a 5-minute task drops to 30 seconds.

Phase 4: Aggregation & Integrity Checks

  • Run a backend script to merge the subagent JSON outputs.
  • Run a read-only integrity gate (e.g., check for dangling edges or malformed schemas). Do not let the LLM do this validation; let the backend code do it.

Phase 5: Handoff

  • Present a clean summary to the user.
  • Offer an interactive follow-up (e.g., "I have generated the artifacts. Would you like me to trace a path through them?").

7. Context Management

Token bloat is the enemy of reliability.

  • The Problem: If a skill asks the host agent to cat a 50,000-line JSON file to verify it, the context window fills, latency skyrockets, and the LLM loses focus.
  • The Solution: Blind Routing. Instruct the agent to run scripts that route data from file to file on disk. The agent should only see aggregate numbers (e.g., Merged 50 nodes) via stdout.
  • Caching: Implement semantic caching in the backend. Before dispatching subagents, the backend compares file hashes and generates a .uncached.txt file. The Orchestrator prompt only feeds .uncached.txt to the subagents.

8. Skill Design Principles

  1. Single Responsibility Paradigm: The LLM is the orchestrator; the backend is the worker. Never ask the LLM to count files, parse complex ASTs, or validate JSON schemas.
  2. Aggressive Validation: Fail fast in backend scripts. If a user provides an invalid URL, the backend script should throw an exit code 1 immediately, which the Orchestrator Prompt sees and relays.
  3. Honesty Rules: Explicitly define what the LLM cannot do. (e.g., "Never invent a connection. If unsure, mark as AMBIGUOUS.")
  4. Deterministic Fallbacks: If the user lacks an API key for a complex semantic step, design the architecture to fall back to a purely structural, local analysis. Never brick the skill entirely.
  5. Idempotency: Ensure that running the skill twice produces the same result or safely updates existing caches without duplication.

9. Patterns Observed

  • The "Fast Path" Shortcut: A rule at the very top of the prompt. "If output/data.json exists AND the user is just asking a question, skip Steps 1-5 and go straight to Query Mode." This prevents unnecessary, expensive rebuilds.
  • The Sandbox Escape: Host agents often run commands in unpredictable working directories. Successful skills dynamically calculate PROJECT_ROOT=$(pwd) and force all paths to be absolute in the bash blocks provided to the agent.
  • Ghost Deduplication: When combining AST data (local) with Semantic LLM data, use predictable ID generation (e.g., path_stem_symbol_name) so backend scripts can deduplicate nodes silently, preventing the LLM from hallucinating duplicates.

10. Building Your Own Skill (Roadmap)

Step 1: Identify the Hybrid Workflow Find a task that mixes CLI commands with semantic reasoning. (e.g., "Audit all security vulnerabilities, then write a remediation PR for each").

Step 2: Build the Deterministic Backend Write the Python/Go scripts that do the heavy lifting. Create a scan(), parse(), and merge() function.

Step 3: Draft the Subagent Spec If you need parallel LLM processing, write a strict JSON-schema prompt for the subagents.

Step 4: Write the Orchestrator Prompt Draft the SKILL.md. Number the steps. Provide the exact bash blocks the agent must run to trigger your backend scripts.

Step 5: Add Always-On Nudges Create an install command that drops a hint in the user's AGENTS.md so the skill becomes part of the permanent project memory.

Step 6: Test Across Frameworks Run it in Cursor, Claude Code, and Aider. Note where tool APIs differ, and build a prompt compiler to template the differences.

11. Generic Skill Template

To bootstrap a new skill, adopt this directory structure:

my-new-skill/
├── .github/workflows/         # CI/CD for backend
├── backend/                   # Traditional code
│   ├── src/                   # Source logic
│   └── pyproject.toml         # Dependencies
├── prompts/                   # The LLM Instructions
│   ├── base_orchestrator.md   # The main step-by-step flow
│   ├── schema_subagent.md     # Instructions for parallel workers
│   └── integration_hook.md    # Text to inject into user rule files
├── scripts/
│   └── build_skill.py         # Compiles prompts/ into final SKILL.md variants
└── README.md

12. Skill Development Checklist

  • Define Objective: What exactly does the skill achieve?
  • Offload Logic: Is any string manipulation or parsing being done by the LLM that could be a Python script?
  • Strict Outputs: Does the subagent prompt demand strict JSON without markdown formatting (```json)?
  • Token Diet: Have you removed large cat or echo commands from the prompt's bash blocks?
  • State Persistence: Are intermediate steps written to a .cache dir on disk?
  • Fast Path: Can the skill answer questions using previous run data without rebuilding?
  • Platform Parity: Have you tested the dispatch logic on platforms that don't support parallel Agent tools?

13. Advanced Design Ideas

  • The "Triage" Agent: Instead of parallelizing blindly, step 2 could be a fast LLM pass that ranks files by relevance, and step 3 only dispatches heavy subagents to the top 20%.
  • Graph RAG Integration: Have the skill automatically push its structured JSON output into a local Neo4j or FalkorDB instance via docker, enabling the host agent to write Cypher queries instead of reading JSON.
  • Self-Healing Bash: Add instructions in the prompt that say: "If the backend script fails with a missing dependency, run pip install or uv tool install and try exactly one more time before failing."
  • Continuous Observability: Log all subagent token usage and execution times to a local .skill_telemetry.json file to monitor cost across large monorepos.

14. Lessons Learned

  1. Do not let the LLM dictate control flow. If you tell an LLM "Extract data, then merge it", it will often try to merge it using its own context window. You must explicitly say "Run python merge.py".
  2. Subagents must be general-purpose. If you restrict a subagent to "Read Only" (e.g., an Explore tool), it cannot write its chunk artifact to disk, breaking the parallelization map-reduce pattern.
  3. The UX relies on the final handoff. A skill shouldn't just dump a file and exit. The best skills end with an interactive hook: "I found 3 critical bugs. Which one should we fix first?"

15. Glossary

  • Host Agent: The primary AI coding assistant interface the user interacts with (e.g., Claude Code, Cursor).
  • Subagent: An ephemeral, parallelized LLM worker spawned by the Host Agent to handle a chunk of a larger task.
  • Orchestrator Prompt: The primary set of instructions (usually SKILL.md) that defines the state machine and steps for the Host Agent.
  • Always-On Instructions: System prompt nudges injected persistently into a workspace (like AGENTS.md) to permanently alter how the Host Agent approaches the project.
  • Prompt Fragment: Modular markdown files containing specific instructions or schemas, stitched together at build time by a tool like skillgen.
  • Blind Routing: An architectural pattern where the LLM is instructed to pass file paths between CLI scripts without ever reading the file contents into its own context window.