Skip to content

quick-start.md

Latest commit

 

History

History
273 lines (199 loc) · 7.04 KB

quick-start.md

File metadata and controls

273 lines (199 loc) · 7.04 KB
Raw

Quick Start Guide

Installation

Using uv (Recommended)

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone and setup
git clone https://github.com/cisco-ai-defense/skill-scanner
cd skill-scanner

# Install all dependencies
uv sync --all-extras

Using pip

# Install the package
pip install cisco-ai-skill-scanner[all]

Basic Usage

Environment Setup (Optional)

# For LLM analyzer and Meta-analyzer
export SKILL_SCANNER_LLM_API_KEY="your_api_key"
export SKILL_SCANNER_LLM_MODEL="anthropic/claude-sonnet-4-20250514"

# For VirusTotal binary scanning
export VIRUSTOTAL_API_KEY="your_virustotal_api_key"

# For Cisco AI Defense
export AI_DEFENSE_API_KEY="your_aidefense_api_key"

See Configuration Reference for every available environment variable.

Interactive Wizard

Not sure which flags to use? Run skill-scanner with no arguments to launch the interactive wizard:

skill-scanner

It walks you through selecting a scan target, analyzers, policy, and output format step by step.

Scan a Single Skill

# From source (with uv)
uv run skill-scanner scan evals/skills/safe-skills/simple-math

# Installed package
skill-scanner scan evals/skills/safe-skills/simple-math

By default, scan runs the core analyzers: static + bytecode + pipeline.

Scan Multiple Skills

# Scan all skills in a directory
skill-scanner scan-all evals/skills --format table

# Recursive scan with detailed markdown report
skill-scanner scan-all evals/skills --format markdown --detailed --output report.md

Demo Results

The project includes test skills in evals/skills/ for evaluation and testing:

[OK] simple-math (SAFE)

$ skill-scanner scan evals/skills/safe-skills/simple-math
============================================================
Skill: simple-math
============================================================
Status: [OK] SAFE
Max Severity: SAFE
Total Findings: 0
Scan Duration: 0.12s

[FAIL] multi-file-exfiltration (CRITICAL)

$ skill-scanner scan evals/skills/behavioral-analysis/multi-file-exfiltration --use-behavioral
============================================================
Skill: config-analyzer
============================================================
Status: [FAIL] ISSUES FOUND
Max Severity: CRITICAL
Total Findings: 11
Scan Duration: 0.37s

Findings Summary:
  CRITICAL: 3
      HIGH: 3
    MEDIUM: 4
       LOW: 1
      INFO: 0

Detected Threats:

  • Data exfiltration (HTTP POST to external server)
  • Reading sensitive files (~/.aws/credentials)
  • Environment variable theft (API_KEY, SECRET_TOKEN)
  • Command injection (eval on user input)
  • Base64 encoding + network exfiltration pattern

Useful Commands

# List available analyzers
skill-scanner list-analyzers

# Validate rule signatures
skill-scanner validate-rules

# Get help
skill-scanner --help
skill-scanner scan --help

See CLI Command Reference for detailed flags and options for every command.

Output Formats

See Output Formats Reference for sample outputs and a format decision guide.

JSON (for CI/CD)

skill-scanner scan /path/to/skill --format json --output results.json

SARIF (for GitHub Code Scanning)

skill-scanner scan /path/to/skill --format sarif --output results.sarif

Markdown (human-readable report)

skill-scanner scan /path/to/skill --format markdown --detailed --output report.md

Table (terminal-friendly)

skill-scanner scan-all evals/skills --format table

Advanced Features

Scan Policies

Use built-in presets or a custom policy to tune detection sensitivity:

# Use a stricter preset
skill-scanner scan /path/to/skill --policy strict

# Use a more permissive preset
skill-scanner scan /path/to/skill --policy permissive

# Generate a custom policy YAML to edit
skill-scanner generate-policy -o my_policy.yaml

# Interactive policy configurator (TUI)
skill-scanner configure-policy

See Scan Policy Guide for full details.

Enable All Analyzers

skill-scanner scan /path/to/skill \
  --use-behavioral \
  --use-llm \
  --use-trigger \
  --use-aidefense \
  --use-virustotal

LLM provider note: --llm-provider currently accepts anthropic or openai. For Bedrock, Vertex, Azure, Gemini, and other LiteLLM backends, set provider-specific model strings and environment variables (see Dependencies and LLM Providers).

Cross-Skill Analysis

skill-scanner scan-all /path/to/skills --check-overlap

Lenient Mode

Tolerate malformed skills (missing fields, non-string descriptions) instead of failing. When SKILL.md is absent, lenient mode falls back to scanning .md files in the directory as instruction bodies — enabling support for non-Codex/Cursor formats such as Claude Code .claude/commands/*.md:

skill-scanner scan /path/to/skill --lenient
skill-scanner scan-all /path/to/skills --recursive --lenient

# Scan a Claude Code commands directory (no SKILL.md)
skill-scanner scan .claude/commands/deploy --lenient

# Use a custom metadata filename instead of SKILL.md
skill-scanner scan /path/to/skill --skill-file README.md

Pre-commit Hook

Using the pre-commit framework (recommended):

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/cisco-ai-defense/skill-scanner
    rev: v1.0.0  # use the latest release tag
    hooks:
      - id: skill-scanner

Or install the built-in hook directly:

skill-scanner-pre-commit install

The hook only scans skill directories with staged changes. Use --all to scan everything.

Next Steps

  1. Review the documentation:

  2. Try scanning your own skills:

    skill-scanner scan /path/to/your/skill

  3. Integrate with CI/CD:

    skill-scanner scan-all ./skills --fail-on-severity high
# Exit code 1 if findings at or above HIGH severity

    See GitHub Actions Integration for a ready-made reusable workflow.

Troubleshooting

UV not found

Install UV:

curl -LsSf https://astral.sh/uv/install.sh | sh

Module not found errors

Sync dependencies:

uv sync --all-extras

Permission errors

UV manages its own virtual environment - no need for manual venv activation.