Skip to content

fix(#621): supervisor status injection during in-flight tool call wedges interactive session - #622

Merged
HenryLach merged 3 commits into
mainfrom
fix/621-defer-batch-end-epilogue-past-inflight-tool
Aug 27, 2026
Merged

HenryLach merged 3 commits into
mainfrom
fix/621-defer-batch-end-epilogue-past-inflight-tool

Conversation

@HenryLach

Copy link
Copy Markdown
Owner

Fixes #621.

Problem

The supervisor batch-end epilogue appends display banners via pi.sendMessage(msg, { triggerTurn: false }). In pi's sendCustomMessage, { triggerTurn: false } always takes the branch that immediately appends a custom entry to the session tree at the current leaf — even while the interactive agent is streaming. When a tool call is in flight (assistant tool_use appended, tool_result not yet appended), that append splices a user-role custom message between the tool_use and its tool_result. The next request then fails with:

Anthropic 400: messages.N.content.M: unexpected `tool_use_id` found in `tool_result`
blocks ... Each `tool_result` block must have a corresponding `tool_use` block in the
previous message.

which permanently wedges the interactive session (every retry re-sends the same broken ordering). Reproduced repeatedly in a real session, under both auto and supervised integration modes — the supervisor-integration-skipped banner is gated on (mode === "supervised" || mode === "auto") && phase !== "completed", so autonomy level is not the lever.

Fix

Gate the batch-end epilogue on interactive-agent idleness:

  • SupervisorNoticeGate (supervisor-dispatch.ts): runs the epilogue immediately when idle (leaf is terminal → safe, banner renders now), else defers it to the next agent_settled boundary — the first lifecycle point at which all tool results, retries, compaction, and queued continuations have finished. Session-scoped, generation-tagged (a newer batch invalidates stale pending work), coalescing, and disabled on shutdown.
    • deliverAs:"nextTurn" is deliberately not used: it pushes into the next turn's in-memory context only, without persisting the entry or emitting message_start/end, so display banners would be silently dropped.
  • stopBatchMonitoring() extracted from transitionToRoutingMode() and called eagerly when deferring, so a heartbeat-timer send cannot splice during the defer window.
  • Wiring in extension.ts: one agent_settled flush (re-checks isIdle() — another extension may have started a run), one session_shutdown clear, invalidate() + generation bump on new batch, and both batch-end callbacks (doOrchStart / doOrchResume) routed through a shared runSupervisorBatchEndEpilogue() helper.

Tests

extensions/tests/supervisor-dispatch.test.ts (9 cases): idle→immediate, busy→defer, settle→flush, busy-settle→no flush, generation supersede, invalidate(), coalescing (latest wins), dispose(), flush-once.

Verification

  • npm run typecheck clean
  • biome clean on changed/new files (pre-existing warnings elsewhere untouched)
  • Full extensions test suite: 3732 pass / 0 fail / 1 skipped, including the existing auto-integration + batch-summary suites (106) and the 9 new gate tests

Design

Design validated with a second-opinion review: agent_settled + a session-scoped, generation-aware dispatcher is the safest available design given the current pi API (there is no pi-native "persist+display a custom entry only at a safe turn boundary" mode).

Follow-up (not in this PR)

  • Other background { triggerTurn: false } sends can splice via the same mechanism and should also route through the gate: heartbeat/status notices, and integration progress/result messages.
  • extension.ts still registers the session_end event, which appears obsolete in current pi (only session_shutdown is defined); the gate uses session_shutdown. Worth reconciling the existing cleanup handler separately.

…621)

The supervisor batch-end epilogue appends display banners via
pi.sendMessage(..., {triggerTurn:false}). In pi's sendCustomMessage,
{triggerTurn:false} always takes the branch that immediately appends a
`custom` entry to the session tree at the current leaf — even while the
interactive agent is streaming. When a tool call is in flight (assistant
tool_use appended, tool_result not yet appended), that append splices a
user-role custom message between the tool_use and its tool_result. The
next request then fails with an Anthropic 400 ("tool_result must have a
corresponding tool_use in the previous message"), permanently wedging the
session. Reproduced repeatedly, under both auto and supervised modes (the
integration-skipped banner fires for either).

Fix: gate the batch-end epilogue on interactive-agent idleness.

- Add SupervisorNoticeGate (supervisor-dispatch.ts): runs the epilogue
  immediately when idle (leaf is terminal — safe, banner shows now), else
  defers it to the next `agent_settled` boundary, when all tool_results
  have been appended. Session-scoped, generation-tagged (a newer batch
  invalidates stale pending work), coalescing, and disabled on shutdown.
  deliverAs:"nextTurn" is unusable here — it neither persists nor displays
  the entry, so banners would be silently dropped.
- Extract stopBatchMonitoring() from transitionToRoutingMode() and call it
  eagerly when deferring, so a heartbeat timer send can't splice during the
  defer window.
- Wire the gate in extension.ts: one agent_settled flush (re-checks
  isIdle() — another extension may have started a run), one session_shutdown
  clear, invalidate + generation bump on new batch, and route both batch-end
  callbacks (doOrchStart / doOrchResume) through it via a shared
  runSupervisorBatchEndEpilogue() helper.
- Tests: supervisor-dispatch.test.ts (idle-immediate, busy-defer,
  settle-flush, busy-no-flush, generation supersede, invalidate, coalesce,
  dispose, flush-once).

Follow-up (not in this change): other background {triggerTurn:false} sends
(heartbeat/status, integration progress/result) can splice via the same
mechanism and should also route through the gate. Also note extension.ts
still registers the (now-obsolete in current pi) `session_end` event; the
gate uses `session_shutdown`.
#621)

Defense-in-depth follow-up to the batch-end epilogue gate. The gate only
covers the batch-end monitor callbacks, but the supervisor has many other
background pi.sendMessage(..., {triggerTurn:false}) sites (integration
progress/result, heartbeat, routing) that can splice a `custom` message
between an assistant tool_use and its tool_result the same way. Observed in
the wild: `supervisor-batch-summary` injected during an in-flight
`orch_integrate` tool call (the integration-result path), which the gate
does not wrap.

Rather than gate every send site, repair the ORDERING of the outgoing
message array on the pi `context` event, which fires before every provider
request (transformContext, on the pi-internal AgentMessage[] before
convertToLlm). repairToolResultOrdering() pulls each assistant's tool
results to immediately follow it (in tool-call order) and relocates any
spliced-in custom/user messages to after the tool-result group. The request
is therefore always valid regardless of where a stray message was appended,
so a mistimed injection from ANY source can no longer wedge the session.

- context-repair.ts: pure repairToolResultOrdering() (returns the same array
  when already well-formed; never mutates input); registered as a `context`
  handler in extension.ts.
- Only transforms the per-request context — the persisted session tree is
  untouched, so it is idempotent and self-correcting across reloads.
- Tests: context-repair.test.ts (well-formed no-op, single/multi splice,
  parallel-group preservation + intra-group splice, multiple splices,
  unanswered tool_use left as-is, idempotency, tiny arrays).

Together with the epilogue gate this closes the whole class: the gate keeps
the persisted tree clean for the common path, and the context repair
guarantees every outgoing request is valid regardless of send site.
@HenryLach

Copy link
Copy Markdown
Owner Author

Recurrence on a non-batch-end path → added a comprehensive in-memory repair

After the epilogue gate landed, the 400 recurred with supervisor-batch-summary
spliced during an in-flight orch_integrate tool call — the integration-result
summary path, which the gate does not wrap (it only covers the batch-end monitor
callbacks). This confirms the class is broader than the batch-end epilogue: any of
the supervisor's background pi.sendMessage(..., {triggerTurn:false}) sites
(integration progress/result, heartbeat, routing) can splice the same way.

Rather than chase each send site, PR #622 now also repairs ordering on the pi
context event (fires before every provider request, on the pi-internal
AgentMessage[] before convertToLlm). repairToolResultOrdering() pulls each
assistant's tool results to immediately follow it and relocates any spliced-in
custom/user messages to after the tool-result group, so the outgoing request is
always valid regardless of which send site injected the stray message — a
mistimed injection can no longer wedge the session. It only transforms the
per-request context (the persisted tree is untouched), so it is idempotent and
self-correcting across reloads.

Net: the epilogue gate keeps the persisted tree clean for the common path, and the
context repair is the guaranteed safety net for all paths (present and future).
Full suite green (3741 pass / 0 fail); 18 new tests across the gate + repair.

…, changelog

Addresses all findings from the Sage strategy + code reviews of the #621
two-layer fix.

Fix 1 — /orch-resume supersession gap (Sage: real lifecycle gap):
  doOrchStart bumped batchGeneration + invalidated the notice gate on
  restart, but doOrchResume did not. An epilogue deferred mid-tool by the
  prior batch kept the same generation, so if the user resumed before
  agent_settled flushed it, onSettled() matched and fired the stale
  epilogue against the resumed batch. Both paths now call a shared
  supersedeDeferredEpilogue() helper (extracted per Sage follow-up so the
  two entry points can't drift apart again — that drift WAS this bug).

Fix 2 — repairToolResultOrdering hardening (Sage: edge-case robustness):
  - Map<string,T[]> queue instead of last-wins single, so duplicate
    toolResults sharing a toolCallId are never dropped (was silent data
    loss).
  - emitted tracked by message identity (Set<T>) not by id.
  - New owner-index pre-pass repairs the result-before-assistant shape
    (a toolResult whose owning assistant appears later is held and pulled
    forward at the owner).
  - Safety-net final pass guarantees no toolResult is ever dropped.
  - toolUseIds() guards null/undefined.
  No-op reference short-circuit and idempotency preserved.

Fix 3 — CHANGELOG [Unreleased] entry documenting both layers, the Sage
  hardening, and the behavior note that auto-integration may now start at
  the agent_settled boundary (bounded latency) rather than instantly.

Tests: context-repair.test.ts +4 (duplicate-no-data-loss, grouped-dup
no-op, result-before-assistant, compound result-before-assistant+parallel
+splice). supervisor-dispatch.test.ts +2 (FAILURE MODE: resume w/o
invalidate fires stale epilogue; FIXED: resume with supersede drops it).

Full suite 3747 pass / 0 fail / 1 skip. typecheck + lint (baseline) +
format all green. Sage code review: no blocking issues, confidence ~0.9.
@HenryLach HenryLach changed the title fix(supervisor): defer batch-end epilogue past in-flight tool calls (#621) fix(#621): supervisor status injection during in-flight tool call wedges interactive session Aug 27, 2026
@HenryLach

Copy link
Copy Markdown
Owner Author

Updated: Sage review remediations applied (commit 4b2cc8d)

Ran a Sage strategy review + code review of the two-layer fix. Applied all findings:

  1. /orch-resume supersession gap (Sage caught a real lifecycle gap): doOrchStart bumped batchGeneration + invalidated the notice gate on restart, but doOrchResume did not — so an epilogue deferred mid-tool by the prior batch could fire against a resumed batch. Both paths now call a shared supersedeDeferredEpilogue() helper (extracted so they can't drift apart again — that drift was the bug).
  2. repairToolResultOrdering hardening: queue-based result grouping (no data loss on duplicate results sharing a toolCallId), identity-based emitted tracking, result-before-assistant repair, safety-net no-drop guarantee, null-guarded toolUseIds.
  3. CHANGELOG [Unreleased] entry incl. the behavior note that auto-integration may now start at the agent_settled boundary (bounded latency).

Tests: +24 total across context-repair.test.ts (13) and supervisor-dispatch.test.ts (11), including the FAILURE-MODE/FIXED resume-supersession pair and a compound result-before-assistant + parallel-group + splice fixture.

Sage code review verdict: no blocking issues, confidence ~0.9, release-ready.

Validation: typecheck ✅ · lint ✅ (286/671 baseline) · format ✅ · full suite ✅ 3747 pass / 0 fail / 1 skip · CLI smoke ✅

@HenryLach
HenryLach enabled auto-merge August 27, 2026 12:05
@HenryLach
HenryLach merged commit bd85258 into main Aug 27, 2026
1 check passed
@HenryLach
HenryLach deleted the fix/621-defer-batch-end-epilogue-past-inflight-tool branch August 27, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(supervisor): pi.sendMessage status injection during in-flight tool call orphans tool_result → Anthropic 400 wedges interactive session

1 participant