Merge PR #533: Security review outputs — full repo audit @ 646f04a

Author: merge-queue-bot

.claude/skills/mdsmith-security-review/SKILL.md

@@ -81,29 +81,73 @@ This skill runs in two modes. Decide first; if unclear, ask.
    for, per surface. **Read this file before forming conclusions** —
    it is the core of the skill.
 
-4. **Record findings in the structured model.** Capture each finding
-   as a JSON object per the schema in `references/output-formats.md`.
-   One finding = one defect. Assign severity using the rubric below
-   and record `confidence` honestly (`confirmed` only if you traced
-   the code path or built a repro; otherwise `likely` / `tentative`).
-
-5. **Emit the three outputs.** From the single `findings.json`,
-   render the report, the SARIF, and the inline annotations. Use the
-   script — do not hand-write SARIF:
+4. **Record findings in the structured model.** Pick the review's
+   directory first: `docs/security/<YYYY-MM-DD-slug>/` (today's date
+   plus a short scope slug, e.g.
+   `docs/security/2026-06-09-full-repo-audit/`). Author the findings
+   as `<dir>/findings.json` per the schema in
+   `references/output-formats.md`. One finding = one defect. Assign
+   severity using the rubric below and record `confidence` honestly
+   (`confirmed` only if you traced the code path or built a repro;
+   otherwise `likely` / `tentative`).
+
+5. **Emit the three outputs.** From the single findings file,
+   render the report, the SARIF, and the inline annotations into
+   the same directory. Use the script — do not hand-write SARIF:
 
    ```bash
-   go run ./cmd/mdsmith-secreview render findings.json --out-dir <dir>
+   go run ./cmd/mdsmith-secreview render \
+     docs/security/<stem>/findings.json \
+     --out-dir docs/security/<stem>/
    ```
 
-   It writes `security-review.md`, `findings.sarif`, and
-   `inline-annotations.json`; see `references/output-formats.md` for
-   the schema and what each output is for.
-
-6. **Summarize honestly.** Lead with the highest-severity confirmed
+   It writes `report.md`, `findings.sarif`, and
+   `inline-annotations.json` beside the `findings.json` input. The
+   schema and the purpose of each output live in
+   `references/output-formats.md`. The per-audit directory keeps
+   every review's files apart, so a later review never overwrites an
+   earlier one. `SECURITY.md`'s catalog indexes the new `report.md`
+   on the next fix pass.
+
+   Then make the report pass the linter: add the front matter
+   `docs/security/proto.md` requires (`date`, `scope`, `method`,
+   `title`, `summary`), keep the H1 directly after the closing
+   `---`, and run `mdsmith fix` on it. Regenerate `SECURITY.md`'s
+   catalog with `mdsmith fix SECURITY.md`.
+
+6. **Schedule the fixes as plans.** A review that finds defects
+   but leaves no track record of how they get fixed is half a
+   review. Turn the actionable findings into `plan/` files so the
+   fix work is queued like any other task. Group by the fix, not by
+   the finding: one plan covers one coherent change even when it
+   closes several findings (the
+   [`83_security-hardening-batch`](../../../plan/83_security-hardening-batch.md)
+   plan is the canonical multi-finding batch).
+
+  - File one plan per Critical/High/Medium finding (or per shared
+     fix). Batch the Low/informational/hardening items into a single
+     "security hardening batch — `<date>`" plan rather than one file
+     each.
+  - Use the next free numeric prefix in `plan/` and follow
+     [`plan/proto.md`](../../../plan/proto.md): front-matter `id`,
+     `title`, `status: "🔲"`, a `summary`, and a `model` suggestion;
+     body sections Goal, Tasks (red/green TDD steps), and Acceptance
+     Criteria.
+  - In the Goal, name each finding it closes by id and link the
+     review's `report.md`, so the plan and the audit cross-reference.
+  - Run `mdsmith fix PLAN.md` to refresh the index, then
+     `mdsmith check plan/` so the new files pass.
+  - In **PR-review mode**, prefer fixing a finding in the PR itself;
+     file a plan only for follow-up work that is out of the PR's
+     scope. In **audit mode**, file plans for every actionable
+     finding.
+
+7. **Summarize honestly.** Lead with the highest-severity confirmed
    findings. Separate confirmed defects from hardening suggestions.
-   If you could not reach a conclusion on an in-scope area (e.g. you
-   couldn't find the recipe-execution code), say so explicitly
-   rather than implying it's clean.
+   List the plan files you filed. If you could not reach a
+   conclusion on an in-scope area (e.g. you couldn't find the
+   recipe-execution code), say so explicitly rather than implying
+   it's clean.
 
 ## Severity rubric
 

.claude/skills/mdsmith-security-review/evals/README.md

@@ -65,18 +65,22 @@ Run these from the repo root.
 3. Start a fresh session with the skill available and give it the
    case `prompt`.
 4. Follow the skill: read `SKILL.md`, then
-   `references/threat-model.md`, trace the code, record
-   `findings.json`, and render it:
+   `references/threat-model.md`. Trace the code. Pick a review
+   directory under `docs/security/`. Record the findings in its
+   `findings.json` and render them:
 
    ```bash
-   go run ./cmd/mdsmith-secreview render findings.json --out-dir out
+   go run ./cmd/mdsmith-secreview render \
+     docs/security/<stem>/findings.json \
+     --out-dir docs/security/<stem>/
    ```
 
 5. Score the mechanical bar with the grader, then judge the prose
    `must` / `must_not` yourself:
 
    ```bash
-   go run ./cmd/mdsmith-secreview grade --findings findings.json \
+   go run ./cmd/mdsmith-secreview grade \
+     --findings docs/security/<stem>/findings.json \
      --cases .claude/skills/mdsmith-security-review/evals/cases.yaml \
      --case <case-id>
    ```

.claude/skills/mdsmith-security-review/evals/cases.yaml

@@ -17,8 +17,8 @@ cases:
         - "Reads references/threat-model.md before concluding."
         - "Confirms the §0 baseline defenses against the actual code (recipes not executed,
            symlink default-deny, VS Code untrustedWorkspaces, no npm postinstall)."
-        - "Produces all three outputs via the script (findings.sarif, security-review.md,
-           inline-annotations.json)."
+        - "Produces all three outputs via the script (findings.sarif, report.md,
+           inline-annotations.json) under docs/security/<stem>/."
         - "Names residual un-traced areas in the coverage note rather than omitting them."
       must_not:
         - "Reports a Critical/High recipe-execution RCE without a traced exec call path."

.claude/skills/mdsmith-security-review/references/output-formats.md

@@ -1,12 +1,31 @@
 # Output Formats
 
-All three outputs are rendered from one **`findings.json`** by the
+All three outputs are rendered from one **findings file** by the
 `mdsmith-secreview render` command. Author that one file; never
 hand-write SARIF.
 
+## File layout
+
+Each review owns a directory `docs/security/<YYYY-MM-DD-slug>/`
+holding four fixed-name files:
+
+```text
+docs/security/<stem>/findings.json             # authored input
+docs/security/<stem>/report.md                 # rendered report
+docs/security/<stem>/findings.sarif            # rendered SARIF
+docs/security/<stem>/inline-annotations.json   # rendered annotations
+```
+
+Render with `--out-dir docs/security/<stem>/`. The directory — not a
+filename stem — namespaces each review, so the basenames are fixed
+and a later review never overwrites an earlier one. `SECURITY.md`'s
+`<?catalog?>` over `docs/security/*/report.md` indexes the report;
+run `mdsmith fix SECURITY.md` after rendering. The `security-note`
+kind validates `report.md` against `docs/security/proto.md`.
+
 ## The finding object
 
-`findings.json` is `{"target": {...}, "findings": [ <finding>, ... ]}`.
+The findings file is `{"target": {...}, "findings": [ <finding>, ... ]}`.
 
 ```json
 {
@@ -73,7 +92,7 @@ title, the CWE (as a `properties.tags` entry), and the
 `security-severity`. `confidence` and `severity` are recorded in
 result `properties`.
 
-## Output 2 (human report): `security-review.md`
+## Output 2 (human report): `report.md`
 
 Layout the renderer produces:
 

.mdsmith.yml

@@ -232,8 +232,8 @@ overrides:
       paragraph-structure:
         max-sentences: 12
         max-words-per-sentence: 120
-  - glob: ["docs/security/*.md", "!docs/security/proto.md",
-           "website/content/docs/security/*.md"]
+  - glob: ["docs/security/*/report.md",
+           "website/content/docs/security/*/report.md"]
     rules:
       line-length:
         max: 600
@@ -378,7 +378,7 @@ kinds:
       required-structure:
         schema: .claude/skills/proto.md
   security-note:
-    path-pattern: "docs/security/{proto.md,[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-*.md}"
+    path-pattern: "docs/security/{proto.md,*/report.md}"
     rules:
       required-structure:
         schema: docs/security/proto.md
@@ -601,8 +601,8 @@ kind-assignment:
   # against a synced file resolves to the same kind as the
   # source. The recursive walk still skips these because the
   # output directory is gitignored.
-  - glob: ["docs/security/*.md",
-           "website/content/docs/security/*.md"]
+  - glob: ["docs/security/proto.md", "docs/security/*/report.md",
+           "website/content/docs/security/*/report.md"]
     kinds: [security-note]
   - glob: ["docs/development/secret-rotations/*.md",
            "website/content/docs/development/secret-rotations/*.md"]

PLAN.md

@@ -169,4 +169,6 @@ footer: |
 | 239 | 🔲     | opus   | [cuelite phase 3 — surface C (row-expr evaluator)](plan/239_cuelite-surface-c.md)                                                       |
 | 240 | 🔲     | sonnet | [cuelite phase 4 — drop cuelang.org and enable tinygo](plan/240_cuelite-drop-cue.md)                                                    |
 | 241 | 🔲     | opus   | [Schema-per-file config under `.mdsmith/schemas/`](plan/241_schema-files.md)                                                            |
+| 242 | 🔲     | sonnet | [Recover from panics in the LSP lint pipeline](plan/242_lsp-panic-recovery.md)                                                          |
+| 243 | 🔲     | sonnet | [Security hardening batch — 2026-06-09 audit](plan/243_secreview-2026-06-09-hardening.md)                                               |
 <?/catalog?>

SECURITY.md

@@ -43,23 +43,25 @@ repository.
 ## Security Audit Log
 
 Point-in-time security reviews live in
-[`docs/security/`](docs/security/) and follow the
-filename pattern `YYYY-MM-DD-<slug>.md`. Each note
-records the scope, the review method, the findings,
-and the fix or follow-up.
+[`docs/security/`](docs/security/). Each review is a
+directory named `YYYY-MM-DD-<slug>/`. It holds a
+`report.md` next to its machine-readable companions:
+`findings.json`, `findings.sarif`, and
+`inline-annotations.json`. The report records the
+scope, the method, the findings, and the follow-up.
 
 <?catalog
 glob:
-  - "docs/security/*.md"
-  - "!docs/security/proto.md"
+  - "docs/security/*/report.md"
 sort: -date
 header: |
   | Date | Review | Scope |
   |------|--------|-------|
 row: "| {date} | [{title}]({filename}) | {scope} |"
 ?>
-| Date       | Review                                                                                                          | Scope                                                                                                                          |
-| ---------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
-| 2026-05-12 | [Supply-Chain Hardening — mini-shai-hulud / TanStack Class](docs/security/2026-05-12-supply-chain-hardening.md) | npm, PyPI, VS Code Marketplace, and Open VSX publishing surface; GitHub Actions CI/CD; lockfile and lifecycle-script handling. |
-| 2026-04-05 | [Adversarial Markdown Input](docs/security/2026-04-05-adversarial-markdown.md)                                  | Adversarial markdown input causing unintended side effects on the host machine                                                 |
+| Date       | Review                                                                                                                 | Scope                                                                                                                          |
+| ---------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
+| 2026-06-09 | [mdsmith security audit — 2026-06-09](docs/security/2026-06-09-full-repo-audit/report.md)                              | full repo — all surfaces                                                                                                       |
+| 2026-05-12 | [Supply-Chain Hardening — mini-shai-hulud / TanStack Class](docs/security/2026-05-12-supply-chain-hardening/report.md) | npm, PyPI, VS Code Marketplace, and Open VSX publishing surface; GitHub Actions CI/CD; lockfile and lifecycle-script handling. |
+| 2026-04-05 | [Adversarial Markdown Input](docs/security/2026-04-05-adversarial-markdown/report.md)                                  | Adversarial markdown input causing unintended side effects on the host machine                                                 |
 <?/catalog?>

cmd/mdsmith-secreview/main.go

@@ -33,8 +33,10 @@ const usageText = `Usage: mdsmith-secreview <command> [args]
 
 Commands:
   render <findings.json> [--out-dir DIR]
-                          Render findings.sarif, security-review.md, and
+                          Render findings.sarif, report.md, and
                           inline-annotations.json into DIR (default ".").
+                          Point DIR at the per-audit directory
+                          docs/security/<YYYY-MM-DD-slug>/.
   grade --findings F (--cases C --case ID | --forbid-severity S...
                       --require-min-severity S --require-location-file F)
                           Grade a findings.json against a case rubric (via
@@ -71,7 +73,7 @@ func run(args []string, stdout, stderr io.Writer) int {
 func runRender(args []string, stdout, stderr io.Writer) int {
 	fs := flag.NewFlagSet("render", flag.ContinueOnError)
 	fs.SetOutput(stderr)
-	outDir := fs.String("out-dir", ".", "directory to write the three outputs into")
+	outDir := fs.String("out-dir", ".", "per-audit directory to write the three outputs into")
 	fs.Usage = func() {
 		_, _ = fmt.Fprint(stderr, "Usage: mdsmith-secreview render <findings.json> [--out-dir DIR]\n")
 	}

cmd/mdsmith-secreview/main_test.go

@@ -66,7 +66,7 @@ func TestRenderHappy(t *testing.T) {
 	require.Equalf(t, 0, code, "stderr: %s", errOut)
 	assert.Contains(t, out, "rendered 1 finding(s) ->")
 	assert.Contains(t, out, "findings.sarif")
-	for _, name := range []string{"findings.sarif", "security-review.md", "inline-annotations.json"} {
+	for _, name := range []string{"findings.sarif", "report.md", "inline-annotations.json"} {
 		_, err := os.Stat(filepath.Join(outDir, name))
 		require.NoErrorf(t, err, "expected %s", name)
 	}

docs/background/markdown-linters.md

@@ -989,7 +989,7 @@ you need a stable rule set while these land.
 [mado]: https://github.com/akiomik/mado
 [panache]: https://panache.bz/
 <!-- mdsmith plan + security + reference links -->
-[mdsmith-sec]: ../security/2026-04-05-adversarial-markdown.md
+[mdsmith-sec]: ../security/2026-04-05-adversarial-markdown/report.md
 [conventions]: ../reference/conventions.md
 [bench]: ../research/benchmarks/README.md
 [mdcov]: ../research/markdownlint-coverage/README.md