Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,11 @@
- Improved dashboard workflow detail with verdict, top blocker, full next actions, and richer JSON fields.
- Added an end-to-end finding lifecycle regression test from template creation through strict reported archive.
- Added copyable candidate, blocked, and confirmed demo Evidence.v1 examples.
- Split the CLI entrypoint into focused command modules and render helpers.
- Extracted CLI usage/help functions into `usage.ts`, reducing `omv.ts` by ~280 lines.
- Added a Python `zipfile` fallback for skill packaging when the system `zip` binary is unavailable.
- Expanded pattern registries to 12 ecosystems (added php, csharp, swift, dart, elixir, perl).
- Expanded `omv-audit` and `omv-repro` eval coverage to 6 scenarios each.
- Added CONTRIBUTING.md with full skill development template and registration guide.

## v0.7.1 - Hardened evidence workflow

Expand Down
118 changes: 118 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,121 @@ Before opening a PR:
2. Update docs when commands, outputs, contracts, or release behavior changes.
3. Update `CHANGELOG.md` for release-facing changes.
4. Explain whether skill output behavior changed.

## Creating a New Skill

Use this template to add a new skill to the collection.

### 1. Directory Structure

```
skills/omv-<name>/
SKILL.md — skill definition (frontmatter name must match directory)
references/ — detailed guidance loaded on demand
patterns/<ecosystem>.md — ecosystem-specific patterns (if applicable)
scripts/
check_output.py — heuristic eval checker
evals/
evals.json — eval scenarios (minimum 3)
golden/ — stable golden outputs
contracts/ — copies of consumed contracts
```

### 2. SKILL.md Frontmatter

```yaml
---
name: omv-<name>
description: |
One paragraph describing when to invoke this skill.
---
```

The `name` field MUST match the directory basename exactly.

### 3. Eval Requirements

Every skill needs at least 3 eval scenarios covering:

- **Happy path** — normal successful invocation
- **Edge case** — boundary condition or unusual input
- **Error/blocked path** — graceful handling of invalid or impossible requests

Each eval in `evals.json` must have:
- `id`: unique integer
- `prompt`: the invocation string
- `expected_output`: human-readable description
- `files`: list of golden output paths
- `assertions`: list of `{type, text}` pairs checked by `check_output.py`

### 4. check_output.py Template

```python
#!/usr/bin/env python3
"""Heuristic checker for omv-<name> eval outputs."""

from __future__ import annotations
import argparse, json, re, sys
from pathlib import Path
from typing import Any

SKILL_DIR = Path(__file__).resolve().parents[1]

def load_eval(evals_path: Path, eval_id: int) -> dict[str, Any]:
data = json.loads(evals_path.read_text(encoding="utf-8"))
item = next((e for e in data["evals"] if e["id"] == eval_id), None)
if item is None:
raise SystemExit(f"unknown eval id: {eval_id}")
return item

def check(assertion_type: str, text: str) -> bool:
# Add assertion checks here
raise SystemExit(f"unknown assertion type: {assertion_type}")

def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--eval-id", type=int, required=True)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--evals", type=Path, default=SKILL_DIR / "evals" / "evals.json")
args = parser.parse_args()
eval_item = load_eval(args.evals, args.eval_id)
output = args.output.read_text(encoding="utf-8")
failures = [
a["type"] for a in eval_item.get("assertions", [])
if not check(str(a["type"]), output)
]
if failures:
print("FAIL: " + ", ".join(failures), file=sys.stderr)
raise SystemExit(1)
print(f"OK: eval {args.eval_id} heuristic assertions passed")

if __name__ == "__main__":
main()
```

### 5. Registration

After creating the skill:

1. Run `python3 scripts/validate_skill.py skills/omv-<name>` to verify structure.
2. Add the skill to `registry.yaml` with version, produces/consumes bindings.
3. Run `python3 scripts/sync_skill_assets.py` to sync shared references.
4. Run `npm test` to verify no regressions.

### 6. Pattern Registry (if applicable)

If your skill uses ecosystem-specific vulnerability patterns, add them to `shared/references/patterns/<ecosystem>.md` with this structure:

```markdown
## <Vuln Class>: <short description>

- Source pattern: ...
- Sink signature: ...
- Common misuse: ...
- Expected guard: ...
- Evidence criteria: ...
- False-positive checks: ...
- CWE: CWE-XXX
```

Currently supported ecosystems: npm, python, go, rust, java, ruby, php, csharp, swift, dart, elixir, perl.
2 changes: 1 addition & 1 deletion scripts/release_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ def validate_pattern_registry() -> None:
"CWE:",
]
root = REPO_ROOT / "shared" / "references" / "patterns"
for ecosystem in ["npm", "python", "go", "rust", "java", "ruby"]:
for ecosystem in ["npm", "python", "go", "rust", "java", "ruby", "php", "csharp", "swift", "dart", "elixir", "perl"]:
path = root / f"{ecosystem}.md"
if not path.exists():
raise SystemExit(f"missing pattern registry: {path.relative_to(REPO_ROOT)}")
Expand Down
33 changes: 33 additions & 0 deletions shared/references/patterns/csharp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# C# Vulnerability Pattern Registry

Use these entries as audit methods. Do not treat them as examples of any specific real package.

## Deserialization: BinaryFormatter/ObjectStateFormatter

- Source pattern: HTTP body, ViewState, cookie, message queue payload, or file content reaches a deserialization call.
- Sink signature: `BinaryFormatter.Deserialize(stream)`, `ObjectStateFormatter.Deserialize(data)`, `NetDataContractSerializer.ReadObject(reader)`.
- Common misuse: untrusted byte stream is deserialized with a formatter that allows arbitrary type instantiation.
- Expected guard: use `System.Text.Json` or `JsonSerializer` with known types, avoid BinaryFormatter entirely, or implement strict `SerializationBinder` with type allowlist.
- Evidence criteria: show untrusted data source, formatter instantiation, Deserialize call, and missing type restriction or binder.
- False-positive checks: data source is trusted internal, custom binder restricts types, formatter is used only for trusted IPC, or code targets .NET 8+ where BinaryFormatter is removed.
- CWE: CWE-502

## Path traversal: Path.Combine

- Source pattern: HTTP parameter, uploaded filename, API input, or config value controls a path segment passed to file operations.
- Sink signature: `Path.Combine(basePath, userInput)`, `File.ReadAllText(path)`, `File.WriteAllBytes(path, data)`.
- Common misuse: `Path.Combine` with an absolute user path ignores the base directory; no canonical path check follows.
- Expected guard: use `Path.GetFullPath` and verify result starts with intended base directory, reject absolute paths and `..` segments.
- Evidence criteria: show user input source, Path.Combine or concatenation, file I/O sink, and missing containment validation.
- False-positive checks: input is validated against allowlist, path is resolved and base-checked, or file operation is read-only on public content.
- CWE: CWE-22

## SSRF: HttpClient with user URL

- Source pattern: HTTP parameter, webhook config, callback URL, or integration setting controls a URL passed to HttpClient.
- Sink signature: `HttpClient.GetAsync(userUrl)`, `HttpClient.SendAsync(request)`, `WebClient.DownloadString(url)`.
- Common misuse: user-controlled URL is fetched without scheme validation, hostname allowlist, or private IP filtering.
- Expected guard: parse URL, enforce https scheme, validate hostname against allowlist, resolve DNS and reject private/loopback ranges, limit redirects.
- Evidence criteria: show URL source, HttpClient call, and missing scheme/host/IP validation.
- False-positive checks: URL is from trusted config, hostname is hardcoded, proxy handles validation, or request is to a fixed internal service.
- CWE: CWE-918
33 changes: 33 additions & 0 deletions shared/references/patterns/dart.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Dart Vulnerability Pattern Registry

Use these entries as audit methods. Do not treat them as examples of any specific real package.

## Path traversal: file serving

- Source pattern: HTTP request path, user-provided filename, or API parameter controls a file path in a server-side Dart application.
- Sink signature: `File(path).readAsBytes()`, `File(path).readAsString()`, `shelf_static` handler with user path.
- Common misuse: user-controlled path segment is joined to a base directory without canonicalization or containment check.
- Expected guard: resolve canonical path, verify it starts with intended base, reject `..` and absolute paths, use `Uri.normalizePath`.
- Evidence criteria: show user input source, path construction, file I/O sink, and missing containment validation.
- False-positive checks: path is from hardcoded asset list, static file handler has built-in traversal protection, or input is validated against allowlist.
- CWE: CWE-22

## SSRF: http.get with user URL

- Source pattern: user input, webhook URL, or external config controls a URL passed to Dart HTTP client.
- Sink signature: `http.get(Uri.parse(userUrl))`, `HttpClient().getUrl(Uri.parse(url))`, `Dio().get(url)`.
- Common misuse: user-controlled URL is fetched without scheme restriction, hostname validation, or private IP filtering.
- Expected guard: parse URI, enforce https scheme, validate hostname against allowlist, resolve DNS and reject private ranges.
- Evidence criteria: show URL source, HTTP client call, and missing scheme/host validation.
- False-positive checks: URL is from trusted config, hostname is hardcoded, or request goes through a validated proxy.
- CWE: CWE-918

## Command injection: Process.run

- Source pattern: user input, filename, or external parameter is interpolated into a shell command string.
- Sink signature: `Process.run(executable, arguments)`, `Process.start(cmd, args)` with shell: true.
- Common misuse: user-controlled string is passed as shell command or unsanitized argument with `runInShell: true`.
- Expected guard: avoid `runInShell: true`, pass arguments as list elements (not interpolated strings), validate input against allowlist.
- Evidence criteria: show user input source, Process.run call with shell mode or string interpolation, and missing input sanitization.
- False-positive checks: arguments are from trusted enum, shell mode is disabled, or input is numeric-only.
- CWE: CWE-78
33 changes: 33 additions & 0 deletions shared/references/patterns/elixir.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Elixir Vulnerability Pattern Registry

Use these entries as audit methods. Do not treat them as examples of any specific real package.

## Code injection: Code.eval_string

- Source pattern: HTTP parameter, WebSocket message, config value, or template variable reaches a dynamic code evaluation function.
- Sink signature: `Code.eval_string(user_input)`, `Code.eval_quoted(ast)`, `:erlang.binary_to_term(data)`.
- Common misuse: user-controlled string is evaluated as Elixir/Erlang code without sandboxing or input restriction.
- Expected guard: avoid dynamic code evaluation entirely, use pattern matching on known commands, or restrict to compile-time macros.
- Evidence criteria: show user input source, eval call site, and missing input validation or sandboxing.
- False-positive checks: input is from admin-only LiveView, eval is compile-time only, or input is validated against a fixed command set.
- CWE: CWE-94

## Atom exhaustion: String.to_atom

- Source pattern: HTTP parameter, JSON key, or external input is converted to an atom without bounds checking.
- Sink signature: `String.to_atom(user_input)`, `:"#{user_input}"`, `List.to_atom(charlist)`.
- Common misuse: unbounded user input creates atoms, which are never garbage collected, leading to VM memory exhaustion.
- Expected guard: use `String.to_existing_atom/1` which raises on unknown atoms, or validate input against a known set before conversion.
- Evidence criteria: show user input source, to_atom call, and missing existing_atom guard or input validation.
- False-positive checks: input is from a fixed enum, to_existing_atom is used, or atom creation is bounded by application logic.
- CWE: CWE-400

## SQL injection: raw Ecto query

- Source pattern: HTTP parameter, search query, or filter value is interpolated into a raw SQL fragment in Ecto.
- Sink signature: `Ecto.Adapters.SQL.query(repo, "SELECT ... #{input}")`, `fragment("... #{input} ...")`.
- Common misuse: user input is string-interpolated into raw SQL fragments instead of using parameterized placeholders.
- Expected guard: use `fragment("... ? ...", ^input)` with pinned variables, or Ecto query builder with automatic parameterization.
- Evidence criteria: show user input source, string interpolation in SQL/fragment, and missing parameterization.
- False-positive checks: input is cast to integer, fragment uses ? placeholders with pinned values, or query builder handles escaping.
- CWE: CWE-89
33 changes: 33 additions & 0 deletions shared/references/patterns/perl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Perl Vulnerability Pattern Registry

Use these entries as audit methods. Do not treat them as examples of any specific real package.

## Command injection: open/system with user input

- Source pattern: CGI parameter, form field, filename, or environment variable is interpolated into a shell command or two-argument open.
- Sink signature: `system("cmd $input")`, `open(FH, "| $input")`, `` `$cmd $input` ``, `exec("$cmd $input")`.
- Common misuse: user-controlled string is interpolated into shell commands without escaping or argument list form.
- Expected guard: use list-form system/exec (`system("cmd", @args)`), avoid shell interpolation, validate input against allowlist.
- Evidence criteria: show user input source, shell interpolation in command string, execution sink, and missing sanitization.
- False-positive checks: input is from trusted source, command uses list form, or input is validated against fixed set.
- CWE: CWE-78

## Path traversal: open with user path

- Source pattern: CGI parameter, uploaded filename, or URL path segment controls a file path in an open call.
- Sink signature: `open(FH, "<$path")`, `open(FH, $path)`, `read_file($path)`.
- Common misuse: user-controlled path is opened without canonicalization, containment check, or null byte filtering.
- Expected guard: canonicalize with `Cwd::realpath`, verify path starts with base directory, reject `..` and null bytes, use three-argument open.
- Evidence criteria: show user input source, open call with user path, and missing containment validation.
- False-positive checks: path is from hardcoded list, realpath check is applied, or file is in read-only public directory.
- CWE: CWE-22

## Regex denial of service: user-controlled pattern

- Source pattern: HTTP parameter, search field, or config value is used as a regex pattern or matched against a vulnerable regex.
- Sink signature: `$input =~ /$user_regex/`, `qr/$user_pattern/`, regex with nested quantifiers on user input.
- Common misuse: user-controlled regex or input matched against exponential-backtracking pattern causes CPU exhaustion.
- Expected guard: use `re::engine::RE2` for user patterns, set match timeout, limit input length, or avoid user-controlled regex.
- Evidence criteria: show user input reaching regex compilation or matching, pattern with catastrophic backtracking potential, and missing timeout/length guard.
- False-positive checks: regex is fixed/hardcoded, input length is bounded, or RE2 engine is used.
- CWE: CWE-1333
33 changes: 33 additions & 0 deletions shared/references/patterns/php.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# PHP Vulnerability Pattern Registry

Use these entries as audit methods. Do not treat them as examples of any specific real package.

## Object injection: unserialize

- Source pattern: HTTP body, cookie, session data, or database value reaches an unserialize call.
- Sink signature: `unserialize($userInput)`, `igbinary_unserialize($data)`.
- Common misuse: attacker-controlled serialized string is deserialized without class allowlist, enabling magic method chains.
- Expected guard: use `json_decode` instead, or pass `allowed_classes: []` option to `unserialize`, or validate input format before deserialization.
- Evidence criteria: show source of serialized data, unserialize call site, available gadget classes with `__wakeup`/`__destruct`, and missing allowed_classes restriction.
- False-positive checks: input is from trusted internal source, allowed_classes is restricted, no exploitable gadget chain exists, or input is validated as JSON.
- CWE: CWE-502

## SQL injection: query interpolation

- Source pattern: HTTP parameter, form field, URL segment, or header value is interpolated into a SQL query string.
- Sink signature: `$pdo->query("... $input ...")`, `mysqli_query($conn, "... $input ...")`, `DB::raw($input)`.
- Common misuse: user input is concatenated or interpolated into SQL without parameterized queries or proper escaping.
- Expected guard: use prepared statements with bound parameters, or ORM query builder with automatic escaping.
- Evidence criteria: show user input source, string interpolation into SQL, query execution sink, and absence of parameter binding.
- False-positive checks: input is cast to integer, query uses prepared statements, input comes from trusted enum, or ORM handles escaping.
- CWE: CWE-89

## Remote code execution: eval/system

- Source pattern: HTTP parameter, uploaded filename, template variable, or config value reaches a code execution function.
- Sink signature: `eval($code)`, `system($cmd)`, `exec($cmd)`, `passthru($cmd)`, `shell_exec($cmd)`, `proc_open($cmd)`, `preg_replace('/e', ...)`.
- Common misuse: user-controlled string is passed to code or command execution without sanitization or allowlisting.
- Expected guard: avoid dynamic code execution entirely, use allowlisted commands with escapeshellarg, or sandbox with restricted function list.
- Evidence criteria: show user input source, path to execution function, and missing input validation or command construction guard.
- False-positive checks: input is from admin-only interface, command is static with no user segments, or execution is disabled by PHP configuration.
- CWE: CWE-78
Loading
Loading