Skip to content

Commit efbbda0

Browse files
feat: add AI readiness audit plugin with marketplace distribution (#101)
* feat: restructure awos-audit as Claude Code plugin with marketplace Set up this repo as a plugin marketplace and restructure the audit skill into a proper Claude Code plugin with: - Orchestrator skill (/awos:audit) that auto-discovers dimensions and coordinates phased parallel execution via the dimension-auditor agent - Generic dimension-auditor agent that runs each dimension in its own context window for thorough analysis - Marketplace manifest at repo root for plugin distribution - Preserved auto-discovery: drop a .md in dimensions/ to extend Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: apply best practices to plugin manifest and agent - Add Write tool to dimension-auditor agent (needed for artifact files) - Improve agent description with explicit delegation trigger phrase - Add license, keywords, and explicit component paths to plugin.json Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove invalid skills/agents fields from plugin manifest These fields are not recognized by the plugin schema validator and cause installation failure. Auto-discovery handles standard directory names (skills/, agents/) without explicit declaration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(audit): skip SEC-03 env template check when no env vars are used SEC-03 was unconditionally failing for modules that don't use environment variables. Now detects env var usage patterns first (process.env, os.environ, dotenv, docker-compose env_file, etc.) and skips the check when none are found. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(audit): make SEC-05 gitignore check stack-aware Instead of checking a hardcoded list of file extensions, SEC-05 now infers relevant sensitive file types from the project's topology. Only flags missing patterns for file types that apply to the detected stack (e.g., *.jks only for Java, service-account*.json only for GCP). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(audit): make DOC-03 API docs check proportional to API surface DOC-03 now assesses API volume and exposure before requiring OpenAPI specs. Small closed APIs (co-located server+client, few endpoints) are skipped. Large public APIs still require formal documentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(audit): rework CLAUDE.md checks as ecosystem-level assessment - AI-01: now evaluates the combined CLAUDE.md ecosystem (root + service + rules), not individual files. Criteria weight what's missing — FAIL for missing fundamentals, WARN for minor gaps. - AI-02 (old service-level existence check): removed, absorbed into AI-01 - Renumber AI-03..AI-08 to AI-02..AI-07 - AI-06: auto-pass when AWOS is detected in the project - AI-07 (old AI-08): reworked from "not bloated" to "meaningful" with 200-line limit per official docs, key test, structure/duplication checks Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(audit): add AI-08 agent feedback loop check New check verifies whether Claude Code has the tools to run and observe the application — browser MCP for web UIs, local invoke for serverless, dry-run for infra, etc. API/CLI/library auto-pass since Bash is built-in. Also refines AI-01 WARN/FAIL criteria to weight what's missing (fundamentals vs minor gaps). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(audit): rewrite SDD dimension as AWOS-specific checks Replace 7 generic SDD checks with 7 AWOS-focused checks: - SDD-01: AWOS installation gatekeeper (all others SKIP if not installed) - SDD-02: Product context completeness (product-definition, roadmap, architecture) - SDD-03: Architecture doc vs codebase drift detection - SDD-04: Feature branches correlate with spec activity - SDD-05: Spec directory triad completeness - SDD-06: Stale/abandoned spec detection - SDD-07: Meaningful agent assignments in tasks Also update E2E-03 to reference new SDD-04 check ID. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(audit): remove redundant AI-06 workflow documentation check AI-06 always auto-PASSed when AWOS was detected, making it redundant with SDD-01 which already gates AWOS installation. Renumber AI-07/08. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(audit): handle plugin-provided MCP servers in AI-04 check AI-04 now checks for enabledPlugins in settings.json as fallback when no .mcp.json is found, since plugins may provide MCP servers that cannot be verified from project files alone. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(installer): register AWOS plugin marketplace in settings.json Add a new installer step that registers the awos-marketplace in the project's .claude/settings.json, so users can discover and install AWOS plugins via `/plugin install` without manual marketplace setup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(audit): add context-aware next steps after audit report Replace the HTML-only post-audit prompt with a unified "What's Next?" step that detects project context (AWOS installed, roadmap exists) and offers relevant options: HTML report, roadmap update/creation, or an AWOS installation tip for non-AWOS projects. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * style: apply prettier formatting Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent fd56414 commit efbbda0

19 files changed

Lines changed: 1419 additions & 1 deletion

.claude-plugin/marketplace.json

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"name": "awos-marketplace",
3+
"owner": {
4+
"name": "Provectus"
5+
},
6+
"metadata": {
7+
"description": "AWOS plugins for AI-native development workflows",
8+
"version": "1.0.0"
9+
},
10+
"plugins": [
11+
{
12+
"name": "awos",
13+
"source": "./plugins/awos",
14+
"description": "Comprehensive code quality audit framework with auto-discoverable dimensions",
15+
"version": "1.0.0"
16+
}
17+
]
18+
}
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"name": "awos",
3+
"version": "1.0.0",
4+
"description": "AWOS code quality audit framework",
5+
"author": {
6+
"name": "Provectus"
7+
},
8+
"license": "MIT",
9+
"keywords": ["audit", "code-quality", "dimensions"]
10+
}

plugins/awos/README.md

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# AWOS Audit Plugin
2+
3+
Extensible, dimension-based code quality audit for Claude Code. Each dimension runs in its own context window for thorough analysis. Run `/awos:audit` and get a scored report with actionable recommendations.
4+
5+
## Install
6+
7+
From the AWOS marketplace:
8+
9+
```bash
10+
claude plugin marketplace add <org>/awos
11+
claude plugin install awos
12+
```
13+
14+
Or directly for local development:
15+
16+
```bash
17+
claude --plugin-dir ./plugins/awos
18+
```
19+
20+
## Usage
21+
22+
Full audit across all dimensions:
23+
24+
```
25+
/awos:audit
26+
```
27+
28+
Single dimension:
29+
30+
```
31+
/awos:audit security
32+
```
33+
34+
## How It Works
35+
36+
Each **dimension** is a self-contained `.md` file in `skills/audit/dimensions/` with YAML frontmatter declaring its dependencies. The orchestrator:
37+
38+
1. Auto-discovers all dimension files and builds a dependency DAG
39+
2. Groups dimensions into execution phases
40+
3. Launches each dimension as a **separate agent** with its own context window (via the `dimension-auditor` agent)
41+
4. Within each phase, all dimensions run **in parallel**
42+
5. Compiles results into a scored report with an overall grade
43+
44+
### Scoring
45+
46+
Every check produces a status: **PASS**, **WARN**, **FAIL**, or **SKIP**. Deductions scale by severity:
47+
48+
| Severity | Max Points | FAIL | WARN |
49+
| -------- | ---------- | ---- | ----- |
50+
| critical | 3 | -3 | -1.5 |
51+
| high | 2 | -2 | -1 |
52+
| medium | 1 | -1 | -0.5 |
53+
| low | 0.5 | -0.5 | -0.25 |
54+
55+
Dimension scores average into an overall percentage mapped to a letter grade (A: 90-100, B: 75-89, C: 60-74, D: 40-59, F: 0-39).
56+
57+
## Dimensions
58+
59+
| Dimension | Severity | Dependencies |
60+
| --------------------------- | -------- | ---------------- |
61+
| **Project Topology** | medium ||
62+
| **Security Guardrails** | critical | project-topology |
63+
| **AI Development Tooling** | high | project-topology |
64+
| **Spec-Driven Development** | critical | project-topology |
65+
| **Documentation Quality** | critical | project-topology |
66+
| **Code Architecture** | high | project-topology |
67+
| **Software Best Practices** | high | project-topology |
68+
| **End-to-End Delivery** | high | all others |
69+
70+
**Project Topology** runs first as a reconnaissance phase — it detects the repo structure, languages, and layers so downstream dimensions can skip irrelevant checks.
71+
72+
**End-to-End Delivery** runs last since it depends on every other dimension's results.
73+
74+
## Outputs
75+
76+
Each audit run writes to `context/audits/YYYY-MM-DD/`:
77+
78+
```
79+
context/audits/YYYY-MM-DD/
80+
├── project-topology.md # per-dimension artifact
81+
├── security.md
82+
├── ...
83+
├── report.md # full audit report
84+
├── recommendations.md # prioritized action items
85+
└── report.html # standalone HTML report (optional)
86+
```
87+
88+
When a previous audit exists, the report includes score deltas per dimension.
89+
90+
## Plugin Structure
91+
92+
```
93+
plugins/awos/
94+
├── .claude-plugin/
95+
│ └── plugin.json # plugin manifest
96+
├── skills/
97+
│ └── audit/
98+
│ ├── SKILL.md # orchestrator skill
99+
│ ├── dimensions/ # auto-discovered dimension files
100+
│ ├── scoring.md # scoring algorithm
101+
│ ├── output-format.md # artifact format spec
102+
│ └── report-template.md # HTML report spec
103+
├── agents/
104+
│ └── dimension-auditor.md # generic agent for any dimension
105+
└── README.md
106+
```
107+
108+
## Extending
109+
110+
Add a new dimension by dropping a `.md` file into `skills/audit/dimensions/`. No other changes needed — the orchestrator auto-discovers it.
111+
112+
```markdown
113+
---
114+
name: my-dimension
115+
title: My Dimension
116+
description: What this dimension measures
117+
severity: high
118+
depends-on: [project-topology]
119+
---
120+
121+
# My Dimension
122+
123+
## Checks
124+
125+
### CHECK-01: Descriptive check name
126+
127+
- **What:** What to verify
128+
- **How:** Commands or instructions to evaluate
129+
- **Pass:** Criteria for PASS
130+
- **Fail:** Criteria for FAIL
131+
- **Severity:** critical | high | medium | low
132+
```
133+
134+
See `skills/audit/SKILL.md` for the full frontmatter schema.
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
---
2+
name: dimension-auditor
3+
description: >-
4+
Audits a codebase against a specific quality dimension. Receives dimension
5+
checks, output format, and optionally a topology summary via the task prompt.
6+
Produces a structured per-dimension artifact with check results, evidence,
7+
and scores. Use when executing individual dimension audits as part of the
8+
/awos:audit workflow or when a single audit dimension needs to run in its
9+
own context window.
10+
tools: Read, Write, Grep, Glob, Bash
11+
---
12+
13+
You are a code quality auditor running a single audit dimension.
14+
15+
## Input
16+
17+
You will receive via the task prompt:
18+
19+
1. **Dimension content** — the full markdown file for one dimension, including check definitions with What/How/Pass/Fail/Warn/Skip-When/Severity fields
20+
2. **Output format** — the per-dimension artifact format specification
21+
3. **Output path** — where to write the artifact (e.g. `context/audits/2025-01-15/security.md`)
22+
4. **Topology summary** (optional) — structured output from the project-topology dimension, provided when this dimension depends on topology results
23+
24+
## Execution
25+
26+
For each check in the dimension:
27+
28+
1. Read the **How** instructions carefully — they describe exactly what to look for (glob patterns, grep searches, file reads)
29+
2. If the check has a **Skip-When** condition, evaluate it first. If the condition is met, mark the check as SKIP
30+
3. Execute the investigation steps described in **How**
31+
4. Compare your findings against the **Pass**, **Fail**, and **Warn** criteria
32+
5. Record the status: PASS, WARN, FAIL, or SKIP
33+
6. Collect concrete evidence: file paths, line numbers, counts, relevant snippets
34+
35+
## Rules
36+
37+
- Follow **How** instructions literally — use the exact glob patterns, grep searches, and file reads specified
38+
- Never invent evidence. If you cannot find what a check looks for, that is a finding (likely FAIL or WARN)
39+
- Keep evidence concise: one line per check, with specific file paths or counts
40+
- If a check references the topology summary and none was provided, mark it SKIP
41+
- Do not modify any project files — this is a read-only audit
42+
43+
## Output
44+
45+
Write the per-dimension artifact to the specified output path using the provided output format. The artifact must include:
46+
47+
- Dimension title, date, score, and grade
48+
- Results table with columns: #, Check, Severity, Status, Evidence
49+
- Any dimension-specific summary data (e.g. topology summary for downstream consumption)
50+
51+
Compute the dimension score using the scoring rules provided in the task prompt.

plugins/awos/skills/audit/SKILL.md

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
---
2+
name: audit
3+
description: >-
4+
Run a comprehensive code quality audit across extensible dimensions. Use when
5+
asked to "audit the code", "run a code audit", "check code quality", "audit
6+
this project", or when the /awos:audit command is invoked. Discovers dimension
7+
files automatically — drop a new .md in dimensions/ to extend. Each dimension
8+
runs in its own context window for thorough analysis.
9+
disable-model-invocation: true
10+
argument-hint: '[dimension-name] or blank for full audit'
11+
---
12+
13+
# Code Audit — Orchestrator
14+
15+
You are the audit orchestrator. Your job is to coordinate dimension-specific auditors, each running in their own context window, and compile results into a final report.
16+
17+
## Step 1 — Discover Dimensions
18+
19+
1. Read all `*.md` files from the `dimensions/` directory (relative to this SKILL.md)
20+
2. Parse YAML frontmatter from each file to extract: `name`, `title`, `severity`, `depends-on`
21+
3. If `$ARGUMENTS` is provided and non-empty, filter to only the dimension whose `name` matches `$ARGUMENTS`. If no match, list available dimensions and stop.
22+
23+
## Step 2 — Build Dependency DAG
24+
25+
1. Build a dependency graph from the `depends-on` fields
26+
2. Group dimensions into execution phases:
27+
- **Phase 1:** Dimensions with no `depends-on` (roots of the DAG)
28+
- **Phase N:** Dimensions whose `depends-on` are all completed in prior phases
29+
3. Phases are computed dynamically — adding or removing dimension files automatically updates the DAG
30+
31+
## Step 3 — Prepare Artifacts Directory
32+
33+
```
34+
context/audits/YYYY-MM-DD/
35+
```
36+
37+
Create this directory. If it already exists, results will be overwritten.
38+
39+
## Step 4 — Check for Previous Audit
40+
41+
1. Scan `context/audits/` for previous audit directories (date-named folders other than today)
42+
2. If a previous audit exists, read its `report.md` to extract per-dimension scores for delta comparison later
43+
44+
## Step 5 — Execute Dimensions
45+
46+
For each execution phase, launch all dimensions in the phase **in parallel** using the Task tool with the `dimension-auditor` agent.
47+
48+
For each dimension, provide the agent with:
49+
50+
1. **The full dimension file content** (read from `dimensions/{name}.md`)
51+
2. **The output format** (read from `output-format.md` in this skill directory — the "Per-Dimension Artifact Format" section)
52+
3. **The scoring rules** (read from `scoring.md` in this skill directory)
53+
4. **The output path:** `context/audits/YYYY-MM-DD/{name}.md`
54+
5. **Topology summary** (for Phase 2+ dimensions): read from `context/audits/YYYY-MM-DD/project-topology.md` — the "Topology Summary" section written by the topology auditor
55+
56+
Wait for all dimensions in a phase to complete before starting the next phase.
57+
58+
### Important
59+
60+
- Launch each dimension as a **separate Task** with `subagent_type: "dimension-auditor"` so each gets its own context window
61+
- Within a phase, launch all Tasks in a **single message** (parallel execution)
62+
- The dimension-auditor agent is read-only — it will not modify project files
63+
64+
## Step 6 — Compile Report
65+
66+
After all dimensions complete:
67+
68+
1. Read all per-dimension artifacts from `context/audits/YYYY-MM-DD/`
69+
2. Compute the overall score: average of all dimension percentages (using the scoring algorithm from `scoring.md`)
70+
3. If a previous audit was found in Step 4, compute per-dimension deltas
71+
4. Compile the full report using the report template from `output-format.md`
72+
5. Write the report to `context/audits/YYYY-MM-DD/report.md`
73+
6. Write prioritized recommendations to `context/audits/YYYY-MM-DD/recommendations.md`
74+
7. Present the full report to the user
75+
76+
## Step 7 — What's Next?
77+
78+
After presenting the report, check the project context and offer next steps using `AskUserQuestion` with `multiSelect: true`.
79+
80+
### Detect context
81+
82+
- **AWOS installed:** `.awos/commands/` directory exists
83+
- **Roadmap exists:** `context/product/roadmap.md` file exists
84+
85+
### Build options
86+
87+
**Always include:**
88+
89+
- "Generate HTML report" — create a standalone HTML version of the audit report
90+
91+
**If AWOS installed + roadmap exists, also include:**
92+
93+
- "Update roadmap with audit findings" — incorporate recommendations into the existing product roadmap
94+
95+
**If AWOS installed + no roadmap, also include:**
96+
97+
- "Create a roadmap informed by audit findings" — start a new roadmap using audit results as input
98+
99+
**If AWOS is NOT installed**, append this note after the question:
100+
101+
> Tip: install AWOS (`npx @provectusinc/awos`) — the best way to make your repo AI-friendly and act on these findings.
102+
103+
### Execute selected options
104+
105+
- **HTML report:** Read the HTML report specification from `report-template.md` in this skill directory. Generate `context/audits/YYYY-MM-DD/report.html` — a single self-contained HTML file (inline CSS, no external dependencies). Include: overall score/grade, per-dimension summary table, detailed checklists, recommendations, issue-only filter toggle.
106+
- **Roadmap (update or create):** Tell the user to run `/awos:roadmap` and reference the audit recommendations at `context/audits/YYYY-MM-DD/recommendations.md` as input.
107+
108+
## Adding New Dimensions
109+
110+
Drop a `.md` file in `dimensions/` with this structure:
111+
112+
```markdown
113+
---
114+
name: my-dimension
115+
title: My Dimension
116+
description: What this dimension measures
117+
severity: high
118+
depends-on: [project-topology]
119+
---
120+
121+
# My Dimension
122+
123+
Brief description.
124+
125+
## Checks
126+
127+
### CHECK-01: Short name
128+
129+
- **What:** What to verify
130+
- **How:** Glob/Grep/Read instructions to evaluate
131+
- **Pass:** Criteria for PASS
132+
- **Fail:** Criteria for FAIL
133+
- **Warn:** (optional) Partial compliance
134+
- **Skip-When:** (optional) Condition to auto-skip
135+
- **Severity:** critical | high | medium | low
136+
```
137+
138+
### Frontmatter Fields
139+
140+
| Field | Required | Description |
141+
| ------------- | -------- | ---------------------------------------------------------------------- |
142+
| `name` | yes | Unique identifier, used for CLI filtering (`/awos:audit my-dimension`) |
143+
| `title` | yes | Human-readable display name |
144+
| `description` | yes | One-line purpose |
145+
| `severity` | yes | Default severity for all checks. Individual checks can override. |
146+
| `depends-on` | no | Dimension `name`s that must complete first. Omit if no dependencies. |

0 commit comments

Comments
 (0)