feat(scouts): isolated scout fleet with hooks, intra-scout subagents, knowledge index, and fleet ops (#46)

* feat: scouts — fleet of focused discovery watchers

Adds a `ralph scout` command that runs a fleet of `watch` discovery
loops, one per focused interest area, with namespaced results and
per-scout manifests. Each scout is an isolated sensor for one topic
cluster: no cross-contamination between, say, ai-security and
ai-eval.

New surface:
  - `ralph scout run [--name <n>]` — deploy one or all scouts
  - `ralph scout ls` — list configured scouts
  - `ralph scout results [name]` — recent runs, per scout or fleet-wide
  - `ralph scout init <name>` — scaffold a new scout with an empty
    manifest

Layout:
  - `scouts/<name>/manifest.json` per scout (same shape as
    `watch-manifest.json`, with its own topics/languages/thresholds)
  - `results/<scout>/pw-YYYYMMDD-HHmm/` — namespaced run output
  - Five scouts seeded: agent-sdks, ai-eval, ai-security, computer-use,
    inference-infra

Under the hood:
  - `SCOUTS_DIR` constant in `src/utils/paths.ts`
  - `runWatch` accepts `manifestPath` and `scoutName` options; results
    namespacing flows from `scoutName`
  - `watch` gains `--manifest` and `--scout` flags so it can be pointed
    at a per-scout manifest directly
  - `readManifest`, `listRuns` exported for the scout command
  - Steering gets a manifest-lifecycle section documenting auto-add,
    freshness tracking, pruning suggestions, and the discoveryLog cap

Gitignore the 105MB compiled `/ralph-for-kiro` binary so it doesn't
sneak into commits; `results/` and `watch-manifest.json` were already
ignored.

Verified: typecheck, lint (biome 2.4.12), 76/76 tests.

* feat(hooks): agentSpawn + stop lifecycle hooks with per-turn sidecars

Wires Kiro CLI's `agentSpawn` and `stop` hooks (added in Kiro 1.24)
into every Ralph agent config so each turn drops a structured
sidecar under the run's `iterations/` directory alongside the
session-derived markdown.

Why: the current runner relies on `<promise>COMPLETE</promise>`
in Kiro's SQLite session history to detect turn end. That works,
but leaves no per-turn audit trail when a run stalls — the
`pw-20260331-0300` run we dug up earlier had `status: running`
and iteration 0 with no way to tell whether kiro-cli ever spawned
an agent at all. The hook sidecars close that gap without replacing
the existing completion check.

Contract for hook scripts — passed via child-process env vars, not
stdin, because the hook's stdin-JSON payload shape varies across
Kiro versions:

  RALPH_RUN_DIR     absolute path to `results/<scout>/pw-*` or
                    `results/pw-*` (for non-scout watches)
  RALPH_ITERATION   1-based iteration this kiro-cli invocation runs
  RALPH_SCOUT_NAME  scout name, empty string for non-scout watches

Scripts:
  - `.kiro/hooks/on-agent-spawn.sh` — writes
    `iterations/NN-spawn.json` at turn start
  - `.kiro/hooks/on-stop.sh`         — writes
    `iterations/NN-turn.json` at turn end

Implementation:
  - `KiroClient.runChat(prompt, hookEnv?)` layers RALPH_* onto
    `process.env` and spawns kiro-cli with the merged env
  - `LoopConfig` gains nullable `runDir` + `scoutName` fields
  - `runWatch` threads `resultsPath` + `scoutName` into the loop
    config
  - `installHookScripts()` helper stamps the two hook scripts
    chmod 0o755 under `.kiro/hooks/`; both `ralph init` and
    `ralph watch init` call it so new installs ship with hooks
  - Hook scripts bundled as text via Bun's loader; `*.sh` module
    decl added to types

Tests: 8 new in tests/hooks.test.ts covering env build, script
installation + idempotency + executable bits, and LoopConfig
round-trip for runDir/scoutName. 84/84 pass, typecheck clean,
biome clean.

* feat(scouts): per-scout .kiro/ tree with OS-process isolation

Each scout now gets its own `.kiro/` tree under `scouts/<name>/.kiro/`
— agents, steering, hooks, settings, session history — and kiro-cli
is spawned with cwd = `scouts/<name>/` so Kiro picks up the per-scout
config instead of the shared repo-root one. Scouts on divergent
topics (ai-security, ai-eval, ...) cannot cross-contaminate because
they never share a session, a steering doc, or a SQLite DB.

Changes:
  - `ensureScoutKiroTree(scoutDir)` helper (src/core/scout-init.ts)
    stamps the per-scout tree idempotently on every run, reusing
    the existing agent JSON + hook installer. Agent config is
    re-written on each run so bumps propagate; steering is
    preserved once customized.
  - `LoopConfig` gains `scoutCwd` (nullable). When set, the loop
    runner passes it through to `KiroClient.runChat` as the spawn
    cwd.
  - `KiroClient.runChat` signature widened from (prompt, hookEnv?)
    to (prompt, { hookEnv?, cwd? }) — the subprocess cwd override
    goes through this path.
  - `runWatch` detects scout runs, resolves an absolute results
    path (hooks must write there regardless of subprocess cwd),
    calls `ensureScoutKiroTree`, and passes `scoutCwd` into the
    loop config.

Biome note: turned off `complexity/useLiteralKeys` in biome.json
because it conflicts with TS strict's
`noPropertyAccessFromIndexSignature` on `NodeJS.ProcessEnv`. TS
strict is the authority.

Tests: 2 new in tests/scout-init.test.ts covering tree shape,
executable bits, agent/steering content, and idempotency under
user customization. 86/86 pass.

* feat(scouts): intra-scout probe-topic subagent for parallel per-topic dives

Inside a scout, the project-watcher agent now delegates per-topic
deep dives to a new probe-topic subagent via the `use_subagent`
tool's `InvokeSubagents` command — one subagent invocation per
manifest topic, all in parallel. Each probe runs in isolated
context and writes a single Markdown report to
`<run-dir>/probes/<topic>.md`. The parent watcher reads probe
outputs and owns the synthesis step (summary.md, manifest
updates).

Cross-scout contamination stays impossible because the probe-topic
subagent only gets spawned from a scout's kiro-cli subprocess,
which is already isolated by cwd under `scouts/<name>/`. Subagents
share the parent's session — but the parent session only sees one
scout's manifest because of M2a's process boundary.

Config changes:
  - src/data/probe-topic.json — new subagent config, narrow
    allowedTools (read, @brave-search, @Tavily, @exa), points at
    probe-topic.md steering
  - src/data/probe-topic.md — subagent steering: one-topic,
    read-only-manifest, structured report contract
  - src/data/project-watcher.json and .kiro/agents/project-watcher.json
    gain availableAgents=["probe-*"] and trustedAgents=["probe-topic"]
    (Kiro 1.25 glob whitelist — nothing else can be spawned)
  - src/data/watcher-context.md documents the fan-out pattern
    with a copy-pasteable use_subagent InvokeSubagents example;
    skip when topic count ≤ 1

Implementation:
  - ensureScoutKiroTree() now stamps probe-topic.{json,md} into
    each scout's .kiro/{agents,steering}/ alongside project-watcher.
    Steering is preserved on user customization; agent config is
    re-written to propagate defaults.

Grounded the schema via `kiro-cli chat --no-interactive` tool
introspection — the real tool is `use_subagent` with
{command:"InvokeSubagents", content:{subagents:[{query, agent_name?}]}};
agent_name matches the `name` field of the agent JSON.

Tests: scout-init.test.ts extended to assert probe-topic config +
steering land in the scout's .kiro/, and that the parent's
availableAgents/trustedAgents glob whitelist is present. 86/86 pass.

* feat(scouts): per-scout knowledge-base index + scout-cwd-relative resources

Each scout's project-watcher agent now gets a `scout-history`
knowledge-base `resources` entry scoped to
`results/<this-scout>/**/summary.md` — so prior findings feed the
retrieval layer but an `ai-security` scout can't see `ai-eval`'s
history, and vice versa. Bright line stays at the scout boundary.

Also fixes the `resources` paths. The default agent config ships
with paths relative to the repo-root cwd
(`file://../../watch-manifest.json`,
`file://../ralph-loop.local.json`). When scout isolation puts kiro-cli
in `scouts/<name>/`, those are wrong. scout-init rewrites them to
`file://manifest.json` and `file://.kiro/ralph-loop.local.json`
before stamping the per-scout config, and the repo-root fallback
keeps its old paths intact.

Steering: adds a short section documenting Kiro 1.26 `@path` inline
refs as the cheaper alternative to `fs_read` for known files
(`@manifest.json`, `@.kiro/ralph-loop.local.json`), and explains the
`scout-history` knowledge resource as the natural-language
retrieval path for past iterations.

Decision deferred: global compaction settings
(`compaction.excludeContextWindowPercent` etc.) live in
`~/.kiro/settings/cli.json` per Kiro's 1.24 docs — they're a user-home
setting, not an agent-config field, so they can't be isolated per
scout via cwd. Out of scope for the wrapper; users who want them run
`kiro-cli settings compaction.excludeContextWindowPercent 20` once
at their home level.

Tests: scout-init.test.ts asserts the knowledge-base entry shape,
type="knowledgeBase", and that `include` is scoped to the scout's
own results/ subtree. 86/86 pass.

* feat(scouts): fleet ops — parallel run, status, tail

Three fleet-level commands for operating multiple isolated scouts:

  - `ralph scout run --concurrency N` — drain pattern with N
    parallel workers, each pulling the next scout off a shared queue
    and running it to completion before pulling the next. One
    scout's failure is caught per-item so the fleet doesn't abort.
    Default concurrency=1 preserves the old sequential behavior.

  - `ralph scout status` — one line per scout showing the latest
    run's task ID, status, iteration count, wall-clock duration,
    and repos-discovered count. Fleet-wide glance.

  - `ralph scout tail <name>` — follow the in-flight run of a
    named scout by polling the `iterations/` directory for new
    hook sidecars (NN-spawn.json / NN-turn.json from M1). Prints
    a timestamped line per new sidecar, and stops automatically
    when `status.json` flips out of `running`. Configurable poll
    interval via `--interval <ms>` (default 2000).

Each parallel worker still runs its scout as a separate kiro-cli
child process with its own cwd under `scouts/<name>/` (M2a), so
concurrency does not compromise isolation — each worker is a fully
independent OS process.

Tests: 4 new in tests/scout-concurrency.test.ts covering the
drain pattern — sequential timing at concurrency=1, parallel
speedup at concurrency=N, one-failure-doesn't-abort invariant,
and input-order preservation in results regardless of finish
order. 90/90 pass.

* fix(tests): drop TOCTOU access+stat pattern flagged by CodeQL

CodeQL's js/file-system-race rule fires when a test calls access(p)
then opens/stats the same path — the file may be deleted between the
two calls. Replaced with a single stat() that throws ENOENT if the
file doesn't exist, which fails the test cleanly without the race.

* fix(tests): use Bun.file() for single-handle reads to clear CodeQL TOCTOU

CodeQL's js/file-system-race fires whenever the same path is passed to
two separate fs calls (stat + readFile, access + stat, ...). The tests
legitimately want to check mode AND content of a file the test itself
just wrote. Switching to Bun.file(path) gives a single handle from
which we can derive stat() and text()/json(), which satisfies the rule
without weakening the assertions.

Author: theagenticguy

.gitignore

@@ -642,3 +642,6 @@ watch-manifest.json
 .kiro/settings/mcp.json
 .kiro/ralph-loop.local.json
 .kiro/ralph-session.json
+
+# Compiled single-file binary (105MB) from `bun build --compile`
+/ralph-for-kiro

.kiro/agents/project-watcher.json

@@ -1,20 +1,22 @@
 {
-  "name": "project-watcher",
-  "description": "Iterative deep research agent for discovering trending GitHub repos matching watched topics and languages",
-  "tools": [
-    "read",
-    "write",
-    "@brave-search",
-    "@tavily",
-    "@exa"
-  ],
-  "allowedTools": [
-    "*"
-  ],
-  "includeMcpJson": true,
-  "resources": [
-    "file://../../watch-manifest.json",
-    "file://../ralph-loop.local.json"
-  ],
-  "prompt": "file://../steering/watcher-context.md"
+	"name": "project-watcher",
+	"description": "Iterative deep research agent for discovering trending GitHub repos matching watched topics and languages. Fans out per-topic deep dives to the probe-topic subagent.",
+	"tools": ["read", "write", "@brave-search", "@tavily", "@exa"],
+	"allowedTools": ["*"],
+	"includeMcpJson": true,
+	"resources": [
+		"file://../../watch-manifest.json",
+		"file://../ralph-loop.local.json"
+	],
+	"prompt": "file://../steering/watcher-context.md",
+	"availableAgents": ["probe-*"],
+	"trustedAgents": ["probe-topic"],
+	"hooks": {
+		"agentSpawn": [
+			{ "command": ".kiro/hooks/on-agent-spawn.sh" }
+		],
+		"stop": [
+			{ "command": ".kiro/hooks/on-stop.sh" }
+		]
+	}
 }

.kiro/hooks/on-agent-spawn.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+# Kiro `agentSpawn` hook — fired by kiro-cli when the agent starts a new turn.
+#
+# ralph-for-kiro sets these env vars on the subprocess before invoking kiro-cli
+# chat, so hooks always know where to write artifacts without parsing the
+# hook's JSON stdin payload:
+#
+#   RALPH_RUN_DIR       Absolute path to this run's `results/<scout>/pw-*/`
+#                       directory. Always exists.
+#   RALPH_ITERATION     1-based iteration number for this kiro-cli invocation.
+#   RALPH_SCOUT_NAME    Scout name (empty for non-scout watch runs).
+#
+# The hook writes an iteration-marker JSON so the runner never has to grep
+# stdout or re-read Kiro's SQLite DB to know a turn started.
+
+set -euo pipefail
+
+RUN_DIR="${RALPH_RUN_DIR:-}"
+ITERATION="${RALPH_ITERATION:-0}"
+
+if [[ -z "${RUN_DIR}" ]]; then
+  exit 0
+fi
+
+mkdir -p "${RUN_DIR}/iterations"
+
+# Pad to 2 digits so `ls iterations/` sorts chronologically.
+ITER_PADDED="$(printf '%02d' "${ITERATION}")"
+MARKER="${RUN_DIR}/iterations/${ITER_PADDED}-spawn.json"
+
+cat > "${MARKER}" <<JSON
+{
+  "iteration": ${ITERATION},
+  "spawnedAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
+  "scout": "${RALPH_SCOUT_NAME:-}",
+  "cwd": "$(pwd)"
+}
+JSON

.kiro/hooks/on-stop.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+# Kiro `stop` hook — fired when the agent signals the turn is complete.
+#
+# See on-agent-spawn.sh for the RALPH_* env-var contract. This hook writes a
+# per-turn `NN-turn.json` sidecar with a simple "turn ended" marker the loop
+# runner can watch for. Kiro versions vary in what they pipe on stdin for this
+# hook; we intentionally do not parse it — the env vars are enough.
+
+set -euo pipefail
+
+RUN_DIR="${RALPH_RUN_DIR:-}"
+ITERATION="${RALPH_ITERATION:-0}"
+
+if [[ -z "${RUN_DIR}" ]]; then
+  exit 0
+fi
+
+mkdir -p "${RUN_DIR}/iterations"
+
+ITER_PADDED="$(printf '%02d' "${ITERATION}")"
+MARKER="${RUN_DIR}/iterations/${ITER_PADDED}-turn.json"
+
+cat > "${MARKER}" <<JSON
+{
+  "iteration": ${ITERATION},
+  "completedAt": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
+  "scout": "${RALPH_SCOUT_NAME:-}"
+}
+JSON

.kiro/steering/watcher-context.md

@@ -150,10 +150,55 @@ Trends and patterns observed across discoveries.
 ## Recommended for Watch List
 Repos that meet auto-add thresholds.
 
+## Pruning Suggestions
+Repos with `source: "discovered"` where `lastSeen` > 90 days ago.
+
 ## Open Questions
 Leads for future discovery runs.
 ```
 
+## Manifest Lifecycle — IMPORTANT
+
+You are responsible for keeping `watch-manifest.json` up to date. On your FINAL iteration:
+
+### Auto-Add (Growth)
+When you discover a repo that exceeds the `autoAddStars` threshold AND is not already in the `watch` array:
+- Add it to the `watch` array with these fields:
+  - `repo`: owner/name
+  - `added`: today's date (YYYY-MM-DD)
+  - `source`: "discovered"
+  - `tags`: relevant topic tags from the manifest
+  - `lastSeen`: today's date
+  - `discoveryCount`: 1
+  - `runId`: the task ID from your prompt
+
+### Freshness Tracking
+For every repo already in the `watch` array that you encounter in search results:
+- Update `lastSeen` to today's date
+- Increment `discoveryCount` by 1
+
+### Pruning Suggestions
+For repos where `source` is `"discovered"` and `lastSeen` is more than 90 days ago:
+- Do NOT remove them
+- List them in the "Pruning Suggestions" section of `summary.md` with reasoning
+- Manual entries (`source: "manual"`) are NEVER flagged
+
+### Discovery Log
+Append an entry to the `discoveryLog` array in the manifest:
+```json
+{
+  "runId": "TASK_ID",
+  "date": "YYYY-MM-DD",
+  "reposAdded": N,
+  "reposUpdated": N,
+  "pruningSuggested": N
+}
+```
+Cap the `discoveryLog` at 50 entries (remove oldest if needed).
+
+### Writing the Manifest
+After making changes, write the updated `watch-manifest.json` back to the project root. Preserve existing manual entries exactly as they are.
+
 ## Completion Rules
 
 - **Min iterations (3):** Do NOT signal completion before iteration 3

biome.json

@@ -8,7 +8,10 @@
 	"linter": {
 		"enabled": true,
 		"rules": {
-			"recommended": true
+			"recommended": true,
+			"complexity": {
+				"useLiteralKeys": "off"
+			}
 		}
 	},
 	"assist": { "actions": { "source": { "organizeImports": "on" } } }

scouts/agent-sdks/manifest.json

@@ -0,0 +1,12 @@
+{
+	"version": "1.0",
+	"topics": ["ai-agents", "mcp", "llm-tooling", "developer-tools", "strands-agents"],
+	"languages": ["python", "typescript", "rust"],
+	"watch": [],
+	"thresholds": {
+		"minStars": 50,
+		"autoAddStars": 500,
+		"maxAgeDays": 30
+	},
+	"discoveryLog": []
+}

scouts/ai-eval/manifest.json

@@ -0,0 +1,37 @@
+{
+	"version": "1.0",
+	"topics": ["ai-evaluation", "llm-benchmarks", "agent-testing", "prompt-testing", "red-teaming", "rag-evaluation"],
+	"languages": ["python", "typescript"],
+	"watch": [
+		{
+			"repo": "confident-ai/deepeval",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["ai-evaluation", "llm-benchmarks", "pytest"]
+		},
+		{
+			"repo": "explodinggradients/ragas",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["rag-evaluation", "ai-evaluation"]
+		},
+		{
+			"repo": "promptfoo/promptfoo",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["prompt-testing", "red-teaming", "ai-evaluation"]
+		},
+		{
+			"repo": "braintrustdata/braintrust-sdk",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["ai-evaluation", "llm-benchmarks"]
+		}
+	],
+	"thresholds": {
+		"minStars": 50,
+		"autoAddStars": 300,
+		"maxAgeDays": 60
+	},
+	"discoveryLog": []
+}

scouts/ai-security/manifest.json

@@ -0,0 +1,31 @@
+{
+	"version": "1.0",
+	"topics": ["ai-safety", "prompt-injection", "llm-guardrails", "ai-red-teaming", "agent-security", "ai-alignment"],
+	"languages": ["python", "typescript", "rust"],
+	"watch": [
+		{
+			"repo": "NVIDIA/NeMo-Guardrails",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["llm-guardrails", "ai-safety"]
+		},
+		{
+			"repo": "guardrails-ai/guardrails",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["llm-guardrails", "ai-safety"]
+		},
+		{
+			"repo": "rebuff-ai/rebuff",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["prompt-injection", "ai-safety"]
+		}
+	],
+	"thresholds": {
+		"minStars": 30,
+		"autoAddStars": 200,
+		"maxAgeDays": 60
+	},
+	"discoveryLog": []
+}

scouts/computer-use/manifest.json

@@ -0,0 +1,31 @@
+{
+	"version": "1.0",
+	"topics": ["browser-automation", "computer-use", "desktop-agents", "gui-grounding", "web-agents", "screen-understanding"],
+	"languages": ["python", "typescript", "rust"],
+	"watch": [
+		{
+			"repo": "anthropics/anthropic-quickstarts",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["computer-use", "desktop-agents"]
+		},
+		{
+			"repo": "browser-use/browser-use",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["browser-automation", "web-agents"]
+		},
+		{
+			"repo": "anthropics/claude-computer-use-demo",
+			"added": "2026-03-23",
+			"source": "manual",
+			"tags": ["computer-use", "desktop-agents"]
+		}
+	],
+	"thresholds": {
+		"minStars": 50,
+		"autoAddStars": 500,
+		"maxAgeDays": 45
+	},
+	"discoveryLog": []
+}