fix(copilot): preserve copilot-home, usage, and permission evidence

Applies the four review findings on PR #22.

The agent-customize CLI parsed --copilot-home but never forwarded it to
the collector, so every CLI run inventoried the real ~/.copilot instead
of the requested path. That broke the CLI half of CHS-AC-6 while the
collector API stayed correct. scripts/agent-customize/cli.mjs now passes
copilotHome through, and a CLI regression test asserts the inventory
resolves under the fixture home and that no evidence path points at the
real user home.

The Copilot platform module dropped the outputTokens that assistant
messages do record. isModelRequestEvent excludes assistant-typed events,
so usage cannot ride the message itself: each assistant message that
reports usage now also emits a companion model.response.completed
carrying outputTokens, messageId as responseId, and requestId. Input and
cache tokens stay absent rather than zero. 10,462 of 11,313 assistant
messages across 491 local sessions carry a non-zero outputTokens, so the
coverage is genuinely partial and the warning is renamed
copilot-per-response-usage-partial.

permission.requested and permission.completed now normalize into the
shared control.permission lifecycle, retaining only requestId, the
request kind, and the result decision; intents, paths, and commands are
never retained. The decision rides the result event alone, because
permissionObservation counts any event carrying a decision and Copilot
emits a request even when policy auto-approves, so decorating both would
double-count and label all 1,177 requests as prompted friction.
toolInvocationId is deliberately left unset: dedupeEvents keys on it,
and 32 toolCallIds repeat across separate permission requests, which
would have silently dropped legitimate events.

Source coverage now emits the canonical session-core-facts fields that
safeSourceCoverage whitelists, keeping the Copilot transcript counters
beside them so analysisWarnings keeps working. Two latent bugs surfaced
here: coverage read probe fields off session descriptors that never
carried them, so withRequest was always zero, and the relevant-set
ladder narrowed even without a requested window, hiding transcript-less
sessions instead of reporting them as unreadable.

CHS-AC-5 and CHS-AC-6 in docs/specs/2026-07-29-copilot-host-support.md
now describe partial per-response usage and the payload-free permission
lifecycle, and Session Diagnostics documents the two added boundaries.

Validated with npm test (874/874), npm run pack:verify (PASS, npm 314 /
runtime zip 338), and node --test test/doc-link-graph.test.mjs (6/6, no
mermaid drift).

Co-authored-by: Copilot App (Claude Opus 5) <223556219+Copilot@users.noreply.github.com>

Author: ShawnXxy

docs/specs/2026-07-29-copilot-host-support.md

@@ -27,9 +27,9 @@ Harness report.
 Copilot's transcript is the richest of any supported host: `events.jsonl`
 records hook, subagent, plan, compaction, and permission lifecycle events
 directly. That evidence must be normalized without inventing coverage Copilot
-does not record. Copilot does not persist per-response token usage in the
-transcript, and VS Code Copilot Chat has no documented durable transcript, so
-both stay explicit boundaries rather than zero values.
+does not record. Copilot records output tokens per assistant message but no
+input tokens, cache tokens, or cost, and VS Code Copilot Chat has no documented
+durable transcript, so both stay explicit boundaries rather than zero values.
 
 ## Acceptance Scenarios
 
@@ -51,14 +51,19 @@ both stay explicit boundaries rather than zero values.
   `--platform copilot` through a capability-owned platform module that reads
   workspace-matching `~/.copilot/session-state/<id>/events.jsonl`, and both the
   public and capability-owned analyzer factories resolve it.
-- **CHS-AC-5 (evidence boundaries):** Copilot facts report
-  `usageFieldsObserved: false` because the transcript records no per-response
-  token usage. Session Diagnostics states that VS Code Copilot Chat has no
-  supported durable transcript and that `session-store.db` is documented as
-  auto-managed and is not an evidence source.
+- **CHS-AC-5 (evidence boundaries):** Copilot facts carry the per-response
+  `outputTokens` the transcript records and omit input tokens, cache tokens, and
+  cost rather than reporting them as zero, so per-response usage coverage is
+  partial and complete usage still requires the opt-in OpenTelemetry export.
+  Permission request and result events normalize into the shared permission
+  lifecycle without retaining prompt intents, paths, or commands. Session
+  Diagnostics states that VS Code Copilot Chat has no supported durable
+  transcript and that `session-store.db` is documented as auto-managed and is not
+  an evidence source.
 - **CHS-AC-6 (bundle propagation):** `harness evidence-bundle --platform copilot`
   freezes a Copilot context and returns all three lanes, and `--copilot-home`
-  routes isolated configuration paths into the Agent Customize lane.
+  routes isolated configuration paths into the Agent Customize lane through both
+  the collector API and the `agent-customize` CLI.
 - **CHS-AC-7 (host routing):** The host adapter matrix carries a Copilot row with
   discovery paths, evidence sources, default output, and a smoke command.
   Portable HTML routing includes Copilot, and Qoder remains the only Canvas host.

references/session-evidence/sessions-diagnostics.md

@@ -259,15 +259,23 @@ transcript records are the primary source: `user.message`, `assistant.message`,
 `tool.execution_start`, `tool.execution_complete`, `hook.start`, `hook.end`,
 `subagent.started`, `subagent.completed`, `session.plan_changed`,
 `session.compaction_start`, `session.compaction_complete`,
+`permission.requested`, `permission.completed`,
 `session.permissions_changed`, and `external_tool.*`. Unrecognized types stay
 explicit `metadata.*` events rather than being dropped or reinterpreted.
 
-Keep three Copilot boundaries explicit:
-
-- Copilot transcripts record no per-response model token usage. Subagent
-  totals (`totalTokens`) and `preCompactionTokens` are aggregates and never
-  become per-response usage. Usage evidence requires the opt-in OpenTelemetry
-  export, which this workflow does not read.
+Keep four Copilot boundaries explicit:
+
+- Copilot records `outputTokens` per assistant message and nothing else. Carry
+  that field as partial per-response usage; never fill input tokens, cache
+  tokens, or cost with zero. Subagent totals (`totalTokens`) and
+  `preCompactionTokens` are aggregates and never become per-response usage.
+  Complete usage evidence requires the opt-in OpenTelemetry export, which this
+  workflow does not read.
+- Permission evidence comes from the `permission.requested` /
+  `permission.completed` pair, joined on `requestId`. Retain only the request
+  kind and the result decision. The prompt intent, paths, and commands are
+  payloads and are never retained. `session.permissions_changed` reports a mode
+  change, not a decision.
 - A session directory can match the workspace and carry no `events.jsonl`. That
   is partial coverage, not zero activity.
 - `~/.copilot/session-store.db` is documented as automatically managed. Never

scripts/agent-customize/cli.mjs

@@ -73,6 +73,7 @@ async function main() {
     codexHome: options["codex-home"],
     claudeHome: options["claude-home"],
     qwenHome: options["qwen-home"],
+    copilotHome: options["copilot-home"],
     claudeStatePath: options["claude-state"] ?? options["claude-state-path"],
     codexAppPath: options["codex-app-path"],
     qoderSharedClientCacheRoot: options["qoder-shared-client-cache-root"] ?? options["shared-client-cache-root"],

scripts/session-analysis/platforms/copilot.mjs

@@ -77,6 +77,25 @@ function resultText(data) {
   return result?.content ?? result?.detailedContent ?? "";
 }
 
+// Copilot permission records carry protocol enums (`read`, `write`, `shell`,
+// `approved`, `denied-interactively-by-user`, ...). The guard keeps unexpected
+// values out of normalized events so prompt text can never leak through a field
+// that is only meant to hold a bounded token.
+const PERMISSION_TOKEN_PATTERN = /^[a-z][a-z0-9._-]{0,63}$/u;
+
+function permissionToken(value) {
+  const token = String(value ?? "").trim().toLowerCase();
+  return PERMISSION_TOKEN_PATTERN.test(token) ? token : null;
+}
+
+function permissionDecisionFor(resultKind) {
+  const token = permissionToken(resultKind);
+  if (!token) return null;
+  if (token.startsWith("approved") || token.startsWith("allowed")) return "allowed";
+  if (token.startsWith("denied") || token.startsWith("rejected")) return "denied";
+  return token;
+}
+
 /**
  * Normalize one Copilot `events.jsonl` record.
  *
@@ -116,6 +135,8 @@ function transcriptEvents(raw, sourceRef, options) {
 
   if (rawType === "assistant.message") {
     const text = typeof data.content === "string" ? data.content : "";
+    const outputTokens = Number(data.outputTokens);
+    const usageObserved = Number.isFinite(outputTokens) && outputTokens >= 0;
     const event = {
       ...base,
       type: "assistant",
@@ -126,8 +147,29 @@ function transcriptEvents(raw, sourceRef, options) {
     };
     if (text) event.userVisibleAssistantMessage = true;
     if (options.includeContent && text) event.content = text;
-    if (data.model) event.model = data.model;
+    // When usage is observed the model is attributed on the companion response
+    // event instead, so a single response is not counted against two events.
+    if (data.model && !usageObserved) event.model = data.model;
     events.push(event);
+    if (usageObserved) {
+      // Copilot reports output tokens per assistant message but never input
+      // tokens or cost, and `isModelRequestEvent` deliberately ignores plain
+      // `assistant` events. Emit a companion response event carrying only the
+      // fields that were actually observed -- omitted token fields read as 0 in
+      // `session-efficiency`, so no input usage or cost is invented here.
+      events.push({
+        ...base,
+        type: "model.response.completed",
+        category: "model",
+        model: data.model ?? null,
+        modelUsage: { outputTokens },
+        usageFieldsObserved: true,
+        responseId: data.messageId ?? null,
+        requestId: data.requestId ?? null,
+        evidenceRef: evidenceRef(sourceRef, "model.response.completed"),
+        summary: "assistant response usage (output tokens only)",
+      });
+    }
     return events;
   }
 
@@ -248,6 +290,39 @@ function transcriptEvents(raw, sourceRef, options) {
     return events;
   }
 
+  if (rawType === "permission.requested" || rawType === "permission.completed") {
+    // Copilot records permission handling as a request/result pair correlated by
+    // `requestId`. Only the correlation id and bounded protocol enums are kept --
+    // the prompt payload (`intention`, `path`, `paths`, `commands`) is dropped.
+    //
+    // The decision rides the result event alone. Attaching one to the request as
+    // well would double-count every permission in the episode summaries, and
+    // Copilot emits a request even when policy auto-approves, so treating the
+    // request as a prompt would overstate friction.
+    const requested = rawType === "permission.requested";
+    const event = {
+      ...base,
+      type: "control.permission",
+      category: "control",
+      lifecyclePhase: requested ? "request" : "result",
+      evidenceRef: evidenceRef(sourceRef, rawType),
+      summary: requested ? "permission requested" : "permission resolved",
+    };
+    // Deliberately not `toolInvocationId`: a tool call can be re-prompted, and
+    // `dedupeEvents` keys on that field, which would drop real observations.
+    if (typeof data.requestId === "string" && data.requestId) {
+      event.permissionRequestId = data.requestId;
+    }
+    const kind = permissionToken(data.permissionRequest?.kind);
+    if (kind) event.permissionKind = kind;
+    if (!requested) {
+      const decision = permissionDecisionFor(data.result?.kind);
+      if (decision) event.permissionDecision = decision;
+    }
+    events.push(event);
+    return events;
+  }
+
   if (rawType === "session.permissions_changed" || rawType === "session.mode_changed") {
     events.push({
       ...base,
@@ -305,6 +380,8 @@ async function probeSessionDirectory(sessionDir, workspace) {
     transcriptAvailable: await pathExists(transcriptPath),
     cwd: descriptor.cwd,
     records: 0,
+    conversationRecords: 0,
+    requestRecords: 0,
     firstSeen: null,
     lastSeen: null,
     workspaceMatch: isWorkspaceMatch(descriptor.cwd, workspace),
@@ -315,6 +392,12 @@ async function probeSessionDirectory(sessionDir, workspace) {
 
   await forEachJsonLine(transcriptPath, (raw) => {
     summary.records += 1;
+    if (raw?.type === "user.message" || raw?.type === "assistant.message") {
+      summary.conversationRecords += 1;
+    }
+    if (raw?.type === "assistant.message" && Number.isFinite(Number(raw?.data?.outputTokens))) {
+      summary.requestRecords += 1;
+    }
     if (raw?.type === "session.start") {
       const data = raw?.data ?? {};
       if (data.sessionId) summary.sessionId = data.sessionId;
@@ -335,16 +418,35 @@ async function probeSessionDirectory(sessionDir, workspace) {
  * A Copilot session directory can exist and match the workspace while carrying
  * no `events.jsonl`. That state stays explicit instead of collapsing into zero
  * activity or a clean result.
+ *
+ * The payload populates the canonical `session-core-facts` transcript fields so
+ * public facts never report unmapped evidence as a confirmed zero. Copilot has
+ * no terminal source, so `terminalOnly` is a measured zero rather than an
+ * unknown, and sessions with a missing or empty transcript surface as
+ * `unreadable`. Copilot-specific counters are kept alongside for warnings and
+ * are dropped by the bounded public schema.
  */
-function buildCopilotSourceCoverage({ scope, roots, matched, inWindow }) {
+function buildCopilotSourceCoverage({ scope, roots, matched, inWindow, inWindowProbes = [] }) {
   const root = roots.find((entry) => entry.kind === "copilot-session-jsonl");
   const workspaceSessions = matched.length;
   const withTranscript = matched.filter((probe) => probe.transcriptAvailable);
   const withoutTranscript = workspaceSessions - withTranscript.length;
+  const timeUnobservedProbes = matched.filter((probe) => !probe.firstSeen && !probe.lastSeen);
   const timeUnobserved = withTranscript.filter((probe) => !probe.firstSeen && !probe.lastSeen).length;
   const emptyTranscripts = withTranscript.filter((probe) => probe.records === 0).length;
   const requestedWindow = scope.sinceTime !== null || scope.untilTime !== null;
 
+  // Mirrors the Cursor precedent. Without a requested window every matched
+  // session stays relevant, so a session whose transcript is missing or empty
+  // surfaces as `unreadable` instead of disappearing from the denominator.
+  let relevant = matched;
+  if (requestedWindow) {
+    relevant = inWindowProbes.length > 0 ? inWindowProbes : timeUnobservedProbes;
+  }
+  const withConversation = relevant.filter((probe) => probe.conversationRecords > 0).length;
+  const withRequest = relevant.filter((probe) => probe.requestRecords > 0).length;
+  const unreadable = relevant.filter((probe) => !probe.transcriptAvailable || probe.records === 0).length;
+
   let status = "observed";
   if (!root?.exists || workspaceSessions === 0) {
     status = "absent";
@@ -361,17 +463,27 @@ function buildCopilotSourceCoverage({ scope, roots, matched, inWindow }) {
     transcript: {
       sourceAvailable: Boolean(root?.exists),
       workspaceSessions,
+      inWindowSessions: inWindow.length,
+      outOfWindowSessions: Math.max(workspaceSessions - inWindow.length - timeUnobservedProbes.length, 0),
+      timeUnobservedSessions: timeUnobserved,
+      relevantSessions: relevant.length,
+      withConversation,
+      withRequest,
+      // Copilot exposes no terminal-only source, so this is measured, not unknown.
+      terminalOnly: 0,
+      unreadable,
+      // Copilot-specific detail retained for adapter warnings.
       withTranscript: withTranscript.length,
       withoutTranscript,
       emptyTranscripts,
-      timeUnobservedSessions: timeUnobserved,
-      inWindowSessions: inWindow.length,
     },
     usage: {
-      // Copilot transcripts carry no per-response model usage. Subagent and
+      // Copilot records output tokens per assistant message but never input
+      // tokens or cost, so per-response usage is partial. Subagent and
       // compaction totals are aggregates and are never projected as per-response
       // usage.
-      perResponseUsageObserved: false,
+      perResponseUsageObserved: withRequest > 0,
+      perResponseUsageFields: ["outputTokens"],
     },
   };
 }
@@ -459,13 +571,15 @@ export class CopilotSessionAnalyzer extends SessionAnalyzer {
       if (probe?.workspaceMatch) matched.push(probe);
     }
 
-    const inWindow = matched
+    const inWindowProbes = matched
       .filter((probe) => probe.transcriptAvailable)
       .filter((probe) => {
         const timestamp = probe.lastSeen ?? probe.firstSeen;
         if ((scope.sinceTime !== null || scope.untilTime !== null) && !timestamp) return false;
         return withinTimeRange(timestamp, scope);
-      })
+      });
+
+    const inWindow = inWindowProbes
       .map((probe) => ({
         sessionId: probe.sessionId,
         workspace: scope.workspace,
@@ -485,7 +599,7 @@ export class CopilotSessionAnalyzer extends SessionAnalyzer {
       }))
       .sort((left, right) => (timestampMillis(right.lastSeen) ?? 0) - (timestampMillis(left.lastSeen) ?? 0));
 
-    scope._copilotSourceCoverage = buildCopilotSourceCoverage({ scope, roots, matched, inWindow });
+    scope._copilotSourceCoverage = buildCopilotSourceCoverage({ scope, roots, matched, inWindow, inWindowProbes });
     return inWindow;
   }
 
@@ -519,8 +633,8 @@ export class CopilotSessionAnalyzer extends SessionAnalyzer {
   async analysisWarnings(scope, _roots, _sessions) {
     const coverage = scope._copilotSourceCoverage;
     const warnings = [{
-      code: "copilot-per-response-usage-unobserved",
-      message: "Copilot transcripts do not record per-response model token usage; usage evidence requires the opt-in OpenTelemetry export.",
+      code: "copilot-per-response-usage-partial",
+      message: "Copilot transcripts record output tokens per assistant response but no input tokens, cache tokens, or cost; complete usage evidence requires the opt-in OpenTelemetry export.",
     }];
     if (!coverage || coverage.status === "absent") {
       warnings.push({

test/agent-customize.test.mjs

@@ -1761,3 +1761,43 @@ test("Copilot inventory without user-home authority keeps project scope only", a
     await rm(fixture.root, { recursive: true, force: true });
   }
 });
+
+test("agent-customize CLI honours --copilot-home instead of the real user home", async () => {
+  // Regression: the CLI advertised --copilot-home but never forwarded it, so an
+  // isolated home still resolved ~/.copilot and scanned the caller's real assets.
+  const fixture = await makeCopilotFixture();
+  try {
+    const result = runAgentCustomizeCli([
+      "inventory",
+      "--provider",
+      "copilot",
+      "--workspace",
+      fixture.workspace,
+      "--copilot-home",
+      fixture.copilotHome,
+    ]);
+
+    assert.equal(result.status, 0, result.stderr);
+    const inventory = JSON.parse(result.stdout);
+
+    // The resolved home must be the override, never the caller's real ~/.copilot.
+    assert.equal(inventory.copilotHome, fixture.copilotHome);
+    assert.notEqual(inventory.copilotHome, path.join(os.homedir(), ".copilot"));
+
+    // The inventory itself must also come from the override.
+    assert.equal(inventory.diagnostics.installedPluginState, "copilot-config");
+    assert.deepEqual(inventory.plugins.map((plugin) => plugin.id), ["acme/delivery"]);
+    assert.deepEqual(
+      inventory.manage.skills.map((skill) => skill.name).sort(),
+      ["review-change", "ship-release", "user-skill"],
+    );
+    for (const item of [...inventory.manage.skills, ...inventory.manage.mcps]) {
+      assert.ok(
+        !item.evidence?.path || !item.evidence.path.startsWith(path.join(os.homedir(), ".copilot")),
+        `inventory item escaped the override home: ${item.evidence?.path}`,
+      );
+    }
+  } finally {
+    await rm(fixture.root, { recursive: true, force: true });
+  }
+});