Add post-run validation gate for batch discovery output

Overnight batch runs have shipped bad data straight to the live site:
undefined slugs, duplicate entries across domain aliases, mismatched
duplicate surfaces, and missing required fields. Add
scripts/batch/validate-results.ts to catch these structurally after a
run — checked against results-full and the merged catalog export —
and wire it into drive.sh as a final gate plus a standalone
`bun run validate:batch` script.

Author: RhysSullivan

AGENTS.md

@@ -9,3 +9,27 @@ another port.
 
 Check `.claude/launch.json` for available server names before starting
 anything — never guess names.
+
+# Batch / catalog generation
+
+After any batch or catalog generation run (`scripts/batch/run-loop.ts`,
+`scripts/batch/drive.sh`, `scripts/batch/export-catalog.ts`, or a manual edit
+to `scripts/batch/results-full/`, `scripts/batch/results/`, or
+`sources/discovered.json`), `bun run validate:batch` must pass before the
+results are treated as done or deployed (loaded into KV via
+`scripts/batch/load-kv.ts`, or exported into `sources/discovered.json` and
+built).
+
+`bun run validate:batch` runs `scripts/batch/validate-results.ts`, which
+checks for the failure modes that have shipped to the live site before:
+slugs/domains that render as literal `/domain/undefined/`, duplicate
+(domain, surface-type, name) entries — within one domain's result, across
+domain aliases, and across the static/discovered merge in
+`sources/discovered.json` — and surfaces missing the fields their detail page
+(`src/pages/[domain]/[surface].astro`) renders unconditionally (name, auth
+status, and a locator: url/spec for http+graphql, url for mcp, command or
+packages for cli).
+
+`drive.sh` already runs this as its final gate and exits nonzero if it fails.
+If you run `run-loop.ts` directly instead of through `drive.sh`, run
+`bun run validate:batch` yourself afterward.

package.json

@@ -8,6 +8,7 @@
     "normalize": "bun scripts/normalize.ts",
     "extract-tools": "bun scripts/extract-tools.ts",
     "validate-favicons": "bun scripts/validate-favicons.ts",
+    "validate:batch": "bun scripts/batch/validate-results.ts",
     "dev": "bun run normalize && astro dev",
     "build": "ASTRO_TELEMETRY_DISABLED=1 bun run normalize && ASTRO_TELEMETRY_DISABLED=1 astro build",
     "preview": "astro preview",

scripts/batch/README.md

@@ -48,6 +48,22 @@ Check the output:
 bun scripts/batch/check-results.ts --dir scripts/batch/results/
 ```
 
+`check-results.ts` scores each domain's discovery quality (grounding, checklist).
+Before treating a run as done or loading it into KV, also run the structural
+gate — it catches problems `check-results.ts` doesn't look for: undefined/empty
+slugs that would render as `/domain/undefined/`, duplicate (domain, type, name)
+entries (within a result, across domain aliases, and across the
+static+discovered merge in `sources/discovered.json`), and surfaces missing
+fields the detail page renders unconditionally:
+
+```sh
+bun run validate:batch
+# or directly:
+bun scripts/batch/validate-results.ts --results-dir scripts/batch/results-full
+```
+
+`drive.sh` runs this automatically as its final step and exits nonzero if it fails.
+
 If a matching corpus file exists, URL grounding uses the corpus. If not, live-loop output is checked against the result evidence URLs, same registrable domain, and detected machine signals.
 
 Load KV:

scripts/batch/drive.sh

@@ -71,3 +71,17 @@ PY
     grep -vE '^dedup:' | tail -3
 done
 echo "=== all batches complete $(date '+%H:%M') ==="
+
+# Final gate: the checklist above scores individual result quality, but it
+# can't see cross-file problems (duplicate/aliased domains, slugs that will
+# render as literal "undefined", static+discovered merge collisions). Run the
+# structural validator over the full results dir before this run is treated
+# as done — a failure here means the data is not safe to load into KV.
+echo "=== validating results $(date '+%H:%M') ==="
+bun scripts/batch/validate-results.ts --results-dir "$OUT" --no-catalog
+VALIDATE_STATUS=$?
+if [ "$VALIDATE_STATUS" -ne 0 ]; then
+  echo "GATE: validate-results failed — fix the offenders above before loading $OUT into KV"
+  exit 4
+fi
+echo "=== validation passed $(date '+%H:%M') ==="