feat(runner): package prompt-template precedence and full resource counting (#189)

The closing pass of issue #189: OQ-019 deferral (b), the package-prompts
hole. Package prompt templates loaded with no precedence enforcement at
all: pi merges package prompt paths first and dedupePrompts is first-wins,
so a staged package's /review silently replaced the repo's own template --
the same inversion skillsOverride closes for skills, live for the one
resource kind that decides what a /name dispatches. That matters doubly
since run.command: the getCommand() pre-check forecloses an UNREGISTERED
/name reaching a same-named template, but a package template shadowing a
protected one was invisible to it. Both halves of what /name runs are now
pinned to reviewed content.

The overlay's prompts/ gains its runtime channel (templates were the one
resource kind with none), merged after discovery so the repo's .pi/prompts
still wins first-path-wins. Repo-beats-package is enforced through the
loader's declared promptsOverride seam; the protected set is PRE-LOADED by
a second, minimal DefaultResourceLoader over the repo then the overlay
prompts dirs, because no per-dir prompt loader is exported at the pin and
the exports map is closed -- pi's own reader through its public surface,
never a hand-rolled parser, built only when packages are staged so the
common job pays nothing.

Counting: packages_loaded grows per-root prompts and themes (the two DATA
kinds a manifest contributes, loaded with no per-root visibility), and a
new post-session commands_registered line reports what each package's
factories actually registered, names and roots only -- a command exists
only once the ExtensionRunner has run, which the loader cannot see. Themes
stay count-only: a theme cannot start work in a headless container, so
precedence enforcement there would be machinery without a failure mode.

Specs: INT-SDK-SESSION-OPTIONS AMENDED, REQ-GLOBAL-PI-OVERLAY AMENDED,
DES-COMMAND-ENTRY-POINT AMENDED (the template half of the fall-through
closure), OQ-019 AMENDED (deferral (b) partially closed, the enablement
mirror half stands on OQ-018). UNCHANGED, checked: INT-CONTAINER-JOB-INPUTS,
INT-PI-PACKAGES-FILE-CONTRACT, REQ-PER-TRIGGER-SKILLS,
REQ-DEPLOYMENT-BOOTSTRAP, DES-OPERATOR-GLOBAL-OVERLAY,
DES-FLOW-RESOLUTION-TWO-ADVISORY-LAYERS, OQ-018, OQ-022.

Signed-off-by: Rob Boerman <robboerman@live.nl>

Author: edgehero

docs/global-pi-overlay.md

@@ -73,6 +73,13 @@ refines but never removes the one above:
 | `extensions/` — by default; skip with `--no-extensions`, each one printed by name | the admin extension (hard-blocked) |
 | `packages/<dir>/` — only with `--with-packages`: what `pi-packages.json` declares **plus what you installed with `pi install`**, exact-pinned either way, staged from npm on **your host** | any package whose name looks like the dispatch admin (hard-blocked); a package a repo declares (never installed, see [`SECURITY.md`](../SECURITY.md)) |
 
+`prompts/` sits in the never-copied column for `import-pi`, but the RUNTIME channel exists (issue #189):
+a `prompts/` directory you place in the overlay by hand loads in every job as prompt templates, with the
+serviced repo's own `.pi/prompts` winning a name collision -- the same repo-beats-overlay rule skills
+follow, and it is enforced against staged packages too, so what a `/name` template means stays reviewed
+content. `import-pi` still refuses to copy your host prompts for you: OQ-019 records why the enablement
+mirror stops at extensions.
+
 The overlay is mounted **read-only** into a container that runs adversarial input, so it must hold **no
 secret**. `import-pi` refuses a `models.json` with a literal `apiKey`, and equally with a literal value
 under an auth-ish provider **header** (a header name carrying `auth`, `api-key`, `token`, `secret` or

docs/workflows.md

@@ -209,6 +209,8 @@ skills, prompts and themes** at once.
 |---|---|---|
 | **extension order** | staged package extensions load **last**, after the repo's and the overlay's | first-path-wins, so nothing a package ships can shadow something you wrote |
 | **skill collisions** | the **repo's** skill wins a name collision against a package's | pi puts package skill paths first, so precedence is re-imposed after the load rather than merely asserted |
+| **prompt collisions** | the repo's `.pi/prompts` template wins against a package's, and the overlay's `prompts/` now loads at all | same inversion, same fix: what a `/name` template means stays reviewed content, which also matters because `run.command` dispatches by `/name` |
+| **counting** | the `packages_loaded` log line reports extensions, skills, prompts and themes per package, and `commands_registered` reports what each package actually registered | a package that contributed nothing is otherwise indistinguishable from one that worked |
 | **the recursion guard** | any extension named like the admin console, or registering a `dispatch_*` tool, is **dropped** and logged | a staged package must not be able to hand the agent the deployment's own control surface |
 | **the overlay is `:ro`** | a package that writes beside itself fails | the overlay is deploy-time config mounted into an adversarial-input container |
 | **secrets** | the overlay must hold none; `doctor` fails if it does | `:ro` is not confidentiality, and job input is untrusted |

image/runner/run-job.mjs

@@ -176,6 +176,11 @@ async function main() {
 				packageRoots: cfg.packages,
 				extensionPaths: resourceLoader.getExtensions().extensions.map((extension) => extension.path),
 				skillPaths: skills.map((skill) => skill.filePath),
+				// The two DATA kinds a manifest contributes (issue #189, OQ-019 (b)): loaded with no
+				// per-root visibility until now, and a package prompt template changes what a /name
+				// dispatches even when it shadows nothing.
+				promptPaths: resourceLoader.getPrompts().prompts.map((prompt) => prompt.filePath),
+				themePaths: resourceLoader.getThemes().themes.map((theme) => theme.filePath),
 			}),
 		});
 	}
@@ -233,6 +238,25 @@ async function main() {
 	// leaving the first call of an extension-provided model unmetered.
 	usageMeter.arm();
 
+	// The one packages count that cannot ride packages_loaded: a COMMAND exists only once the
+	// ExtensionRunner has executed the factories, which is here, after createAgentSession -- the
+	// loader knows extension paths, never what they registered. Names and roots only (operator-staged
+	// config, the packages_loaded discipline); a staged package whose factory registered nothing still
+	// reports [], which is how an operator learns their run.command has nothing to bind to before the
+	// first job of it refuses command-unregistered.
+	if (cfg.packages.length > 0) {
+		const registered = session.extensionRunner.getRegisteredCommands();
+		log("commands_registered", {
+			packages: cfg.packages.map((root) => ({
+				root,
+				commands: registered
+					.filter((command) => owningRoot(command.sourceInfo?.path, cfg.packages) === root)
+					.map((command) => command.name)
+					.sort(),
+			})),
+		});
+	}
+
 	// A command job dispatches BEFORE any spend, so verify the command is actually registered first
 	// (issue #189). pi's fallthrough is the hazard being closed: an unregistered "/name" is not an
 	// error to session.prompt() -- it falls through to prompt-template expansion and then to the

image/runner/src/loader.mjs

@@ -241,6 +241,11 @@ export function buildResourceLoader({
 	// which is exactly why it must not say the opposite of what a job does.
 	allowGlobalExtensions = true,
 	packagePaths = [],
+	// Map<name, PromptTemplate> of the PROTECTED prompt templates (repo, then overlay), pre-loaded by
+	// buildLoadedResourceLoader because the promptsOverride seam below is synchronous and a loader
+	// cannot reload() inside its own override. `null` -- every job without staged packages -- makes
+	// the override the identity function.
+	protectedPrompts = null,
 	settingsManager,
 	log = defaultLog,
 } = {}) {
@@ -252,6 +257,7 @@ export function buildResourceLoader({
 	const globalPersona = readIfExists(`${globalPiDir}/APPEND_SYSTEM.md`);
 	const globalSkills = `${globalPiDir}/skills`;
 	const globalExtensions = `${globalPiDir}/extensions`;
+	const globalPrompts = `${globalPiDir}/prompts`;
 	// Only a local job carries an /outbox mount; a github job has none, so its prompt never
 	// pays for the protocol. Evaluated ONCE here at loader build, not per message, so the
 	// assembled prompt is byte-identical across turns (CONST-PERSONA-IN-CACHED-PREFIX).
@@ -337,10 +343,104 @@ export function buildResourceLoader({
 				packageRoots: packagePaths,
 				protectedRoots: protectedSkillRoots,
 			}),
+		// The operator's overlay prompt templates (issue #189, OQ-019 deferral (b)): the overlay's
+		// skills and extensions already load, prompts were the kind with no channel at all. Gated like
+		// the overlay skills, and merged AFTER discovery, so the repo's own .pi/prompts still wins a
+		// first-path-wins name collision against the overlay -- the persona layers' most-specific-wins
+		// ordering, applied to the last resource kind that lacked it.
+		additionalPromptTemplatePaths: [...(existsSync(globalPrompts) ? [globalPrompts] : [])],
+		// "Repo wins on conflict" for PROMPT TEMPLATES, the same inversion skillsOverride closes for
+		// skills: pi merges package prompt paths FIRST and dedupePrompts is first-wins, so a staged
+		// package's /review template silently replaces the repo's. That is worse for prompts than for
+		// skills in one specific way: a run.command job's pre-dispatch getCommand() check forecloses an
+		// unregistered command falling through to a same-named TEMPLATE, but a template that shadows a
+		// PROTECTED one is invisible to that check -- this override is the second half of that hazard's
+		// closure (DES-COMMAND-ENTRY-POINT).
+		promptsOverride: (loaded) =>
+			enforceProtectedPromptPrecedence(loaded, {
+				packageRoots: packagePaths,
+				protectedPrompts,
+			}),
 		appendSystemPromptOverride: () => [guardrails, outboxProtocol, globalPersona, projectPersona].filter(Boolean),
 	});
 }
 
+/**
+ * REQ-GLOBAL-PI-OVERLAY's "repo wins on conflict", for prompt templates. The shape is
+ * enforceProtectedSkillPrecedence's; the substitute source differs by necessity: pi exports
+ * loadSkillsFromDir but no per-dir prompt loader (the exports map is closed, so the dist module is
+ * unreachable), which is why the protected set arrives PRE-LOADED as a Map -- read by pi's own
+ * loader machinery in loadProtectedPrompts, never by a second hand-rolled parser of a format we do
+ * not own.
+ *
+ * pi's collision diagnostics from the raw load are left exactly as written; the substitution appends
+ * its own, so both stages are on the record. A job with no packages, or one whose protected roots
+ * held no prompts, passes through untouched.
+ */
+export function enforceProtectedPromptPrecedence(base, { packageRoots = [], protectedPrompts = null } = {}) {
+	const prompts = base?.prompts ?? [];
+	const diagnostics = base?.diagnostics ?? [];
+	if (!protectedPrompts || protectedPrompts.size === 0 || packageRoots.length === 0) return { prompts, diagnostics };
+
+	const enforced = [];
+	const resolved = prompts.map((prompt) => {
+		if (!isUnderAnyRoot(prompt.filePath, packageRoots)) return prompt;
+		const winner = protectedPrompts.get(prompt.name);
+		if (!winner || winner.filePath === prompt.filePath) return prompt;
+		enforced.push({
+			type: "collision",
+			message: `name "/${prompt.name}" collision -- protected root wins (REQ-GLOBAL-PI-OVERLAY)`,
+			path: prompt.filePath,
+			collision: {
+				resourceType: "prompt",
+				name: prompt.name,
+				winnerPath: winner.filePath,
+				loserPath: prompt.filePath,
+			},
+		});
+		return winner;
+	});
+
+	return { prompts: resolved, diagnostics: [...diagnostics, ...enforced] };
+}
+
+/**
+ * Load the PROTECTED prompt templates (the repo's .pi/prompts, then the overlay's prompts/) through a
+ * second, minimal DefaultResourceLoader: everything off except explicit prompt paths. This is pi's own
+ * reader reached through its public surface -- the only alternative was a hand-rolled parser of the
+ * template format, which is how two readers drift (the enforceProtectedSkillPrecedence docstring's
+ * argument, hit harder here because no per-dir prompt loader is exported at the pin).
+ *
+ * Returns null -- enforcement off -- when the job stages no packages (the overwhelmingly common path
+ * pays nothing) or when no protected root exists on disk. Root order is precedence: the Map keeps the
+ * FIRST of each name, so a repo template beats an overlay template before a package is ever consulted.
+ */
+export async function loadProtectedPrompts({ cwd = WORKSPACE, globalPiDir = GLOBAL_PI_DIR, packagePaths = [], settingsManager } = {}) {
+	if (packagePaths.length === 0) return null;
+	const roots = [`${cwd}/.pi/prompts`, `${globalPiDir}/prompts`].filter((dir) => existsSync(dir));
+	if (roots.length === 0) return null;
+
+	const mini = new DefaultResourceLoader({
+		cwd,
+		agentDir: getAgentDir(),
+		settingsManager,
+		noContextFiles: true,
+		noExtensions: true,
+		noSkills: true,
+		// Suppresses DISCOVERY only; the explicit paths below still load (the same contract the main
+		// loader leans on for skills, pinned by the loader tests).
+		noPromptTemplates: true,
+		additionalPromptTemplatePaths: roots,
+	});
+	await mini.reload();
+
+	const byName = new Map();
+	for (const prompt of mini.getPrompts().prompts) {
+		if (!byName.has(prompt.name)) byName.set(prompt.name, prompt);
+	}
+	return byName;
+}
+
 /**
  * Build and load. Separate from buildResourceLoader so tests can assert on a
  * half-built loader if they need to, but nothing should skip this.
@@ -354,7 +454,11 @@ export function buildResourceLoader({
  * reload() has no early return: it re-runs the entire load every call. Call it once.
  */
 export async function buildLoadedResourceLoader(options = {}) {
-	const loader = buildResourceLoader(options);
+	// The protected prompt set must exist before the loader does: promptsOverride is a synchronous
+	// seam, and a loader cannot reload() a second loader from inside its own reload. Null on the
+	// package-less path, so the common job builds exactly one loader as before.
+	const protectedPrompts = options.protectedPrompts ?? (await loadProtectedPrompts(options));
+	const loader = buildResourceLoader({ ...options, protectedPrompts });
 	await loader.reload();
 	return loader;
 }

image/runner/src/packages.mjs

@@ -177,11 +177,19 @@ export function findShadowedSkills(diagnostics, { packageRoots = [], protectedRo
  * (an unbuilt extension, a manifest pointing at files that are not there) is otherwise
  * indistinguishable from one that worked, and the job runs without the tools its flow expects.
  */
-export function countPackageResources({ packageRoots = [], extensionPaths = [], skillPaths = [] } = {}) {
+export function countPackageResources({ packageRoots = [], extensionPaths = [], skillPaths = [], promptPaths = [], themePaths = [] } = {}) {
+	// Prompts and themes joined the counts with issue #189 (OQ-019 deferral (b)): a package manifest
+	// contributes all four kinds, and the two data kinds were loading with no per-root visibility at
+	// all -- a template that shadows nothing still changes what /name dispatches, so a package that
+	// shipped one deserves the same "contributed 0 or N" line the code kinds get. Extension COMMANDS
+	// are deliberately not counted here: they exist only once an ExtensionRunner has executed the
+	// factories, which is after createAgentSession -- run-job logs them separately (commands_registered).
 	return packageRoots.map((root) => ({
 		root,
 		extensions: extensionPaths.filter((path) => isUnderRoot(path, root)).length,
 		skills: skillPaths.filter((path) => isUnderRoot(path, root)).length,
+		prompts: promptPaths.filter((path) => isUnderRoot(path, root)).length,
+		themes: themePaths.filter((path) => isUnderRoot(path, root)).length,
 	}));
 }
 

image/runner/test/compose.test.mjs

@@ -143,3 +143,19 @@ test("run-job.mjs wires the command path: env-authoritative prompt, pre-spend ve
 	assert.match(src, /command: cfg\.command \? \{ failed: commandFailed \} : null/, "decideExit must receive the command outcome");
 	assert.match(src, /log\("command_dispatch", \{ command: name \}\)/, "the dispatch line carries the NAME only, never args");
 });
+
+test("run-job.mjs counts all four package resource kinds, and commands after the session exists", () => {
+	// Same source-guard tactic as above. Two facts: (1) packages_loaded feeds prompt and theme paths
+	// into countPackageResources -- the two DATA kinds loaded with no per-root visibility before
+	// issue #189 (OQ-019 (b)); (2) commands_registered fires AFTER createAgentSession, because a
+	// command exists only once the ExtensionRunner has executed the factories -- the loader knows
+	// extension paths, never what they registered.
+	const src = readFileSync(new URL("../run-job.mjs", import.meta.url), "utf8");
+	assert.match(src, /promptPaths: resourceLoader\.getPrompts\(\)/, "packages_loaded must count package prompts");
+	assert.match(src, /themePaths: resourceLoader\.getThemes\(\)/, "packages_loaded must count package themes");
+	assert.ok(
+		src.indexOf("createAgentSession") < src.indexOf('log("commands_registered"'),
+		"commands are countable only post-session",
+	);
+	assert.match(src, /getRegisteredCommands\(\)/, "the count must come from the runner's registry, not the manifest");
+});