Skip to content

Commit e6b345c

Browse files
committed
feat: campaign workflow, pattern packs, provenance, and readiness gates
Ship the Unreleased v0.10 foundation without bumping the published version: - Campaign.v1 with omv first/campaign init|list|show|seed (candidate-only seed) - SourceRef.v1 and report provenance manifests for artifact integrity - 14 PatternPack manifests, R/Lua registries, and omv eval runner - Preflight hygiene: private .omv/ gitignore, clean npm pack, dashboard renderer - Shared report-readiness helpers (isReportReady) used by doctor/review - Maintainer docs aligned; keep local research state and OpenSpec scaffolding out of git
1 parent ace3379 commit e6b345c

128 files changed

Lines changed: 9082 additions & 715 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.npmignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
**/__pycache__/**
2+
**/*.pyc
3+
**/*.pyd
4+
**/*.pyo
5+
**/test_*.py

AGENTS.md

Lines changed: 47 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -9,68 +9,44 @@ The project ships a TypeScript CLI (`omv`) for installing skills, plus Markdown
99
```
1010
src/
1111
cli/
12-
omv.ts — CLI entry point (setup / doctor / findings / help)
13-
setup.ts — copies installable skills to ~/.claude/skills/ or ./.claude/skills/
14-
doctor.ts — checks installation health
15-
findings.ts — creates, lists, validates, and promotes Evidence.v1 handoffs
16-
paths.ts — path utilities (claudeSkillsDir, projectSkillsDir, findingsDir, packageRoot, …)
12+
omv.ts — thin CLI entry (dispatches to commands/)
13+
commands/ — one module per top-level command (findings, campaign, review, …)
14+
findings.ts — Evidence.v1 parse / validate / score / doctor / archive
15+
workflow.ts — shared readiness + next-action policy
16+
review.ts — report-readiness verdicts (ready | needs-*)
17+
campaign.ts — Campaign.v1 first-mile research plans
18+
setup.ts / doctor.ts — install skills+agents; health checks
19+
paths.ts — claudeSkillsDir, findingsDir, packageRoot, …
1720
index.ts — package exports
1821
19-
skills/
20-
omv/SKILL.md — collection manager (/omv)
21-
omv-find/SKILL.md — find and rank audit targets (/omv-find)
22-
omv-find/references/
23-
scoring.md — scoring rubric, confidence adjustments, filtering, LOC estimation
24-
output-contract.md — final table contract, audit tips, invalid-request template
25-
omv-find/scripts/check_output.py — heuristic eval checker
26-
omv-find/evals/evals.json — behavior-focused eval scenarios
27-
omv-find/evals/golden/ — stable golden outputs
28-
omv-report/SKILL.md — generate VulDB/CVE/GHSA/OSV reports (/omv-report)
29-
omv-report/references/
30-
ecosystems.md — vendor/product/version rules, CWE mapping, duplicate-CVE databases
31-
report-templates.md — VulDB, GHSA, OSV JSON, Markdown advisory templates
32-
examples/ — filled advisory examples
33-
omv-report/scripts/check_output.py — heuristic eval checker
34-
omv-report/evals/evals.json — behavior-focused report-generation eval scenarios
35-
omv-report/evals/golden/ — stable golden outputs
22+
skills/ — 9 installable skills (self-contained after setup)
23+
omv, omv-find, omv-audit, omv-repro, omv-report,
24+
omv-radar, omv-dedup, omv-disclose, omv-critic
3625
3726
shared/
38-
references/
39-
ecosystems.md — ecosystem registry sources, GitHub search shapes, flagship exclusions
40-
vuln-patterns.md — vulnerability aliases and source -> sink -> guard patterns
41-
cvss-builder.md — CVSS v3.1 metric decision table and common vectors
42-
scripts/
43-
collect_metadata.py — collects GitHub and selected registry metadata as JSON
44-
estimate_loc.sh — estimates source LOC from a GitHub URL or local checkout
27+
references/ — ecosystems, vuln-patterns, cvss-builder, per-eco patterns/
28+
pattern-packs/ — 14 PatternPack.v1 JSON manifests
29+
scripts/ — collect_metadata, estimate_loc, run_evals, …
4530
4631
contracts/
47-
evidence.v1.yaml — finding object: the typed boundary between omv-find and omv-report
48-
candidate-list.v1.yaml — candidate table entry schema produced by omv-find
49-
threat-map.v1.yaml — dataflow threat map schema (planned: omv-audit M2+)
50-
51-
agents/
52-
vuln-scanner.md — passive candidate discovery
53-
dataflow-tracer.md — source -> sink -> guard analysis
54-
cvss-analyst.md — CVSS v3.1 computation
55-
dedup-analyst.md — duplicate CVE/GHSA search
56-
report-writer.md — platform-specific advisory rendering
57-
guard-checker.md — adversarial guard bypass assessment
58-
verifier.md — adversarial conclusion refutation
59-
60-
.claude/agents/ — Claude Code project subagent registration (auto-discovered)
61-
<name>.md — frontmatter (name, description, tools, model) + system prompt body
62-
Each subagent's body references the matching agents/*.md domain spec. See
63-
docs/architecture/agent-team-upgrade.md for the orchestration design.
32+
evidence.v1.yaml — finding object (find → report boundary)
33+
candidate-list.v1.yaml — omv-find table entries
34+
threat-map.v1.yaml — source → transform → sink graph (omv-audit sidecar)
35+
verification.v1.yaml — adversarial verifier review sidecar
36+
campaign.v1.yaml — research campaign plan + seed lanes
37+
source-ref.v1.yaml / report-provenance.v1.yaml / submission.v1.yaml
38+
39+
agents/ — Claude Code subagent specs (installed by omv setup)
40+
vuln-scanner, dataflow-tracer, guard-checker, cvss-analyst,
41+
dedup-analyst, report-writer, verifier
6442
6543
scripts/
66-
sync_metadata.py — sync package, registry, and README metadata
67-
sync_skill_assets.py — sync canonical shared/contract assets into self-contained skill dirs
68-
validate_skill.py — validates all skill directories and optional .skill packages
69-
package_skill.sh — builds a .skill archive from a skill directory
70-
release_check.py — release-time validator, package builder, SHA-256 manifest printer
71-
72-
registry.yaml — collection metadata: versions, produces/consumes bindings
73-
.github/workflows/validate.yml — CI validation, packaging checks, stable golden evals
44+
sync_metadata.py / sync_skill_assets.py / validate_skill.py
45+
package_skill.sh / release_check.py / pattern_packs.py
46+
47+
registry.yaml — skills, agents, contracts, versions
48+
openspec/ — accepted specs + change archive
49+
.github/workflows/validate.yml
7450
```
7551

7652
## CLI
@@ -86,13 +62,30 @@ npx oh-my-vul setup --dry-run # preview only
8662
omv doctor
8763
omv doctor --json
8864

65+
# Workspace + campaign
66+
omv dashboard
67+
omv first --target <name> --ecosystem npm --vuln traversal --no-interactive
68+
omv campaign list|show|seed <id>
69+
omv review <id> --strict
70+
8971
# Manage project-local Evidence.v1 findings
9072
omv findings list
9173
omv findings init <id>
9274
omv findings init <id> --status candidate|confirmed|blocked --force
9375
omv findings validate
9476
omv findings validate <id|path>
9577
omv findings promote <id|path> --status candidate|confirmed|blocked
78+
omv findings workflow
79+
omv findings doctor <id>
80+
omv findings archive <id> --reason blocked|reported
81+
82+
# Sidecars and release gates
83+
omv threat-map init|validate <id>
84+
omv verification init|validate <id>
85+
omv sources init|validate <id>
86+
omv report artifacts|provenance <id>
87+
omv repro init <id>
88+
omv eval --json
9689
```
9790

9891
Build the CLI:

CHANGELOG.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,15 @@
22

33
## Unreleased
44

5-
- Richer ThreatMap.v1 rendering: `omv findings show` now displays the full `source → transforms → sink` dataflow per path with per-path confidence, bypassable guards, and a summary line. Previously the renderer collapsed each path to a single `[source] -> [sink]` line, discarding transforms, confidence, and the summary block that the producer now writes.
5+
### Planned for v0.10.0 — Campaign + evidence graph (draft)
6+
7+
Ship when release notes and `registry.yaml` / `package.json` versions are bumped together.
8+
9+
- **Campaign.v1 first-mile planning**`omv campaign init|list|show|seed` and the `omv first` alias. Seeding creates conservative candidate Evidence only and never overwrites existing findings or creates proof artifacts.
10+
- **SourceRef.v1 + report provenance**`omv sources init|show|validate` and `omv report provenance` manifests that hash Evidence, reports, and available local dependencies. Missing manifests warn; stale confirmed manifests fail artifact checks.
11+
- **PatternPack.v1 + unified evals** — 14 JSON pattern-pack manifests (including R/Lua), manifest-driven find/audit asset sync, and `omv eval` with human/JSON/JUnit output.
12+
- **ThreatMap rich render**`omv findings show` prints full `source → transforms → sink` paths with confidence, bypassable guards, and summary (no longer collapses to a single source→sink line).
13+
- **Readiness policy helpers**`isReportReady` / `isSubmissionScoreReady` / `resolveDoctorNextAction` in `workflow.ts` as the shared report-readiness gate used by doctor and review; maintainer docs (`AGENTS.md`, `CLAUDE.md`, `SPEC.md` banner) aligned with the current tree.
614

715
## v0.9.0 - CLI command split and local findings dedup
816

CLAUDE.md

Lines changed: 18 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -7,72 +7,32 @@ The project ships a TypeScript CLI (`omv`) for installing skills, plus Markdown
77
## Structure
88

99
```
10-
src/
11-
cli/
12-
omv.ts — CLI entry point (setup / doctor / help)
13-
setup.ts — copies skills/ to ~/.claude/skills/
14-
doctor.ts — checks installation health
15-
paths.ts — path utilities (claudeSkillsDir, packageRoot, …)
16-
index.ts — package exports
17-
18-
skills/
19-
omv/SKILL.md — collection manager (/omv)
20-
omv-find/SKILL.md — find and rank audit targets (/omv-find)
21-
omv-find/references/
22-
scoring.md — scoring rubric, confidence adjustments, filtering, LOC estimation
23-
output-contract.md — final table contract, audit tips, invalid-request template
24-
omv-find/scripts/check_output.py — heuristic eval checker
25-
omv-find/evals/evals.json — behavior-focused eval scenarios
26-
omv-find/evals/golden/ — stable golden outputs
27-
omv-report/SKILL.md — generate VulDB/CVE/GHSA/OSV reports (/omv-report)
28-
omv-report/references/
29-
ecosystems.md — vendor/product/version rules, CWE mapping, duplicate-CVE databases
30-
report-templates.md — VulDB, GHSA, OSV JSON, Markdown advisory templates
31-
examples/ — filled advisory examples
32-
omv-report/scripts/check_output.py — heuristic eval checker
33-
omv-report/evals/evals.json — behavior-focused report-generation eval scenarios
34-
omv-report/evals/golden/ — stable golden outputs
35-
36-
shared/
37-
references/
38-
ecosystems.md — ecosystem registry sources, GitHub search shapes, flagship exclusions
39-
vuln-patterns.md — vulnerability aliases and source -> sink -> guard patterns
40-
cvss-builder.md — CVSS v3.1 metric decision table and common vectors
41-
scripts/
42-
collect_metadata.py — collects GitHub and selected registry metadata as JSON
43-
estimate_loc.sh — estimates source LOC from a GitHub URL or local checkout
44-
45-
contracts/
46-
evidence.v1.yaml — finding object: the typed boundary between omv-find and omv-report
47-
candidate-list.v1.yaml — candidate table entry schema produced by omv-find
48-
threat-map.v1.yaml — dataflow threat map schema (planned: omv-audit M2+)
49-
50-
agents/
51-
vuln-scanner.md — passive candidate discovery
52-
dataflow-tracer.md — source -> sink -> guard analysis
53-
cvss-analyst.md — CVSS v3.1 computation
54-
dedup-analyst.md — duplicate CVE/GHSA search
55-
report-writer.md — platform-specific advisory rendering
56-
57-
scripts/
58-
validate_skill.py — validates all skill directories and optional .skill packages
59-
package_skill.sh — builds a .skill archive from a skill directory
60-
release_check.py — release-time validator, package builder, SHA-256 manifest printer
61-
62-
registry.yaml — collection metadata: versions, produces/consumes bindings
63-
.github/workflows/validate.yml — CI validation, packaging checks, stable golden evals
10+
src/cli/ — TypeScript CLI (commands/ split; findings/workflow/review/campaign domain modules)
11+
skills/ — 9 omv-* skills (find, audit, repro, report, radar, dedup, disclose, critic, manager)
12+
shared/ — references, pattern-packs, eval runner helpers
13+
contracts/ — Evidence, ThreatMap, Verification, Campaign, SourceRef, Submission, …
14+
agents/ — subagent specs installed to ~/.claude/agents/ by omv setup
15+
openspec/ — accepted specs + change archive
16+
registry.yaml — versions and produces/consumes bindings
6417
```
6518

19+
Canonical maintainer map: see `AGENTS.md` (kept in sync with the current tree). Early vision draft `SPEC.md` is historical only.
20+
6621
## CLI
6722

6823
```sh
69-
# Install skills to ~/.claude/skills/
24+
# Install skills + agents
7025
npx oh-my-vul setup
71-
npx oh-my-vul setup --force # overwrite existing
72-
npx oh-my-vul setup --dry-run # preview only
26+
npx oh-my-vul setup --scope project
27+
npx oh-my-vul setup --force
28+
npx oh-my-vul setup --dry-run
7329

74-
# Check installation health
30+
# Health and workspace
7531
omv doctor
32+
omv doctor --strict
33+
omv dashboard
34+
omv review <id> --strict
35+
omv findings workflow
7636
```
7737

7838
Build the CLI:

DEVELOPMENT.md

Lines changed: 44 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -190,21 +190,62 @@ Tradeoff:
190190

191191
The ledger is intentionally lightweight YAML, not a database or platform. That keeps the project easy to install and review, but it means deeper validation and rendering still need future deterministic helpers.
192192

193+
### Current iteration - Manifest-driven PatternPacks and evals
194+
195+
What changed:
196+
197+
- Added one PatternPack.v1 JSON manifest for every supported ecosystem, including R and Lua.
198+
- Made manifests drive skill-local pattern distribution and release-time methodology checks.
199+
- Added one stable eval manifest plus a human/JSON/JUnit Python runner and `omv eval` CLI adapter.
200+
201+
Core idea:
202+
203+
Repeated asset and eval lists are contracts disguised as source code. Moving membership into validated JSON keeps progressive disclosure, package self-containment, local CI, and release checks aligned without rewriting skill-specific assertions.
204+
205+
Tradeoff:
206+
207+
The unified runner still starts one Python process per checker. The suite is small, and preserving each Skill's domain-specific checker is more valuable than premature shared assertion abstractions.
208+
193209
## Current Weaknesses
194210

195211
- The Evidence.v1 contract is copied into runtime skill directories; drift is checked by `scripts/sync_skill_assets.py --check`, but the duplication still adds release-surface noise.
196212
- `omv-find` can guide Evidence.v1 handoff creation, but candidate discovery quality still depends on model discipline, source inspection, and available metadata.
197213
- `omv-report` consumes validation guidance, but advisory rendering is still primarily model-written rather than deterministic.
198214
- Examples are partly synthetic.
199-
- Package archives are tracked but not independently diffable.
200-
- The README is user-facing; keep maintainer workflow details in `CONTRIBUTING.md`, `RELEASE.md`, and this file.
215+
- Subagent orchestration is documented and installable, but still optional prose-driven rather than a forced fan-out runtime (see `docs/architecture/agent-team-upgrade.md`).
216+
- Active skills (`omv-radar`, `omv-dedup`, `omv-disclose`, `omv-critic`) have thinner golden coverage than find/audit/repro/report.
217+
- `src/cli/findings.ts` remains a large domain module (validate + score + doctor + archive).
218+
- Historical `SPEC.md` listed skills that never shipped; treat OpenSpec + `registry.yaml` as truth.
201219

202220
## Proposed Next Iterations
203221

204-
### v0.8 - Real Workflow Walkthrough
222+
### v0.10 - Campaign + evidence graph (in flight / Unreleased)
223+
224+
Goal:
225+
226+
Make the first-mile campaign story and evidence-graph sidecars a coherent release:
227+
228+
- `omv first` / Campaign.v1 seed → candidate queue
229+
- ThreatMap + Verification + SourceRef + provenance on the report path
230+
- `omv review --strict` as the pre-report gate in user docs
231+
- Ship Unreleased items as `v0.10.0`
232+
233+
### v0.11 - Deterministic report compiler
205234

206235
Goal:
207236

237+
Evidence.v1 → structured advisory IR → VulDB/GHSA/OSV/Markdown via CLI render, with LLM only polishing narrative paragraphs.
238+
239+
### v0.12 - Agent team runtime minimum
240+
241+
Goal:
242+
243+
Tighten subagent tools (no unrestricted Bash), default omv-audit orchestration stages, and strict Verification requirements before confirmed/report-ready.
244+
245+
### Earlier: Real Workflow Walkthrough (delivered in docs/examples)
246+
247+
Goal (historical):
248+
208249
Add a sanitized end-to-end example that demonstrates the intended user path:
209250

210251
- run `/omv-find` for a realistic candidate;

README.md

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,12 @@ npx oh-my-vul setup --scope project
3232
## Fast Workflow
3333

3434
```text
35+
omv first --target acme --ecosystem npm --vuln traversal,auth --no-interactive
36+
-> .omv/campaigns/acme.yaml + deterministic runbook
37+
38+
omv campaign seed acme
39+
-> candidate Evidence.v1 hypotheses only
40+
3541
/omv-find --lang npm --vuln traversal --count 10
3642
-> choose a candidate
3743
@@ -47,6 +53,9 @@ omv review <id> --strict
4753
4854
/omv-report <id>
4955
/omv-critic <id>
56+
omv sources init <id>
57+
omv report provenance <id>
58+
omv report artifacts <id>
5059
omv submissions record <id> --platform vuldb --submission-id 12345 --url https://example.test/submission/12345
5160
omv findings archive <id> --reason reported
5261
```
@@ -57,12 +66,17 @@ Use `omv dashboard` or `/omv next` whenever you are unsure what to do next.
5766

5867
```sh
5968
omv dashboard
69+
omv campaign list
70+
omv campaign show <id>
6071
omv findings workflow
6172
omv findings show <id>
6273
omv findings validate <id>
6374
omv review <id> --strict
75+
omv sources validate <id>
76+
omv report provenance <id>
6477
omv report artifacts <id>
6578
omv submissions track <id>
79+
omv eval --json
6680
```
6781

6882
Useful setup and health checks:
@@ -71,14 +85,15 @@ Useful setup and health checks:
7185
omv doctor --strict
7286
omv request preflight
7387
omv version --json
88+
omv eval --junit
7489
```
7590

7691
## Skills
7792

7893
<!-- omv:skills:start -->
7994
| Skill | Command | Category | Purpose |
8095
|---|---|---|---|
81-
| `omv` | `/omv` | manager | Local-first project manager — shows workspace status, active finding next actions, archive state, and installed skills |
96+
| `omv` | `/omv` | manager | Local-first project manager — creates research campaigns, shows workspace status, and delegates finding lifecycle actions |
8297
| `omv-find` | `/omv-find` | research | Find and rank open-source packages worth auditing for passive CVE research |
8398
| `omv-audit` | `/omv-audit` | audit | Deep-audit a candidate finding — prove or disprove the vulnerability, fill Evidence.v1 fields for omv-report |
8499
| `omv-repro` | `/omv-repro` | audit | Guide local reproduction of a finding — walk through execution, record observed_result, confirm or block |
@@ -116,11 +131,14 @@ Project state lives under `.omv/` and is private by default.
116131

117132
| Path | Purpose |
118133
|---|---|
134+
| `.omv/campaigns/<id>.yaml` | Campaign.v1 target, scope, priorities, and lanes |
135+
| `.omv/campaigns/<id>.md` | deterministic campaign runbook |
119136
| `.omv/findings/<id>.yaml` | Evidence.v1 finding ledger |
137+
| `.omv/sources/<id>.yaml` | SourceRef.v1 local source identity and Evidence hash |
120138
| `.omv/threatmaps/<id>.yaml` | ThreatMap.v1 source -> sink -> guard graph |
121139
| `.omv/verifications/<id>.yaml` | Verification.v1 adversarial review result |
122140
| `.omv/repro/<id>/` | local reproduction notes and artifacts |
123-
| `.omv/reports/<id>/` | generated report drafts |
141+
| `.omv/reports/<id>/` | generated report drafts and `provenance.json` input hashes |
124142
| `.omv/submissions/<id>.yaml` | submission tracking |
125143

126144
Create or inspect findings:

0 commit comments

Comments
 (0)