Drop dangling tool calls from a chat turn's history before the model sees it

CHANGELOG.md

@@ -8,6 +8,21 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.
 
 ## Unreleased
 
+### A conversation is no longer stuck after a tool call went unanswered
+
+A tool that runs in the browser can be torn down while its call is still open, most often because
+the tab was closed or reloaded mid-run. The call stayed in the thread with no result, every retry
+sent it back up, and the model API refused the whole conversation with `Tool result is missing for
+tool call ...`. The next three things the person typed failed identically, and the only way out was
+to notice that and start another channel.
+
+A chat turn now drops a tool call nothing is going to answer before the conversation reaches the
+model, which is what routines already did for the history they seed. A result counts as an answer
+only if it arrives before the next thing the person or the system said, matching the rule the model
+API enforces, so a handler that resolves after the person has typed again no longer looks like an
+answer. Ids are never rewritten and the stored thread is untouched, so the transcript still shows
+what happened and a call waiting on a resume still gets its result.
+
 ### Coworkers are made in a wizard and managed in a dialog
 
 Creating a coworker is now a three-step wizard — who it is, who may see it, then where it runs,

server/src/agents/history-sanitize.ts

@@ -0,0 +1,153 @@
+/**
+ * The one filter that keeps a broken conversation from being replayed at a model provider for ever.
+ *
+ * IT LIVES HERE BECAUSE BOTH TURN PATHS NEED IT. A routine's headless turn seeds history itself
+ * (`routines/run-turn.ts`) and a chat turn is handed history by the browser (`copilot.ts`). Both
+ * hand that history to a `BuiltInAgent`, which converts it and lets the model provider validate the
+ * call/result pairing. `run-turn.ts` imports `../copilot`, so this cannot live in either of them
+ * without one importing the other back.
+ */
+import type { Message, ToolCall } from "@ag-ui/client";
+
+/** Whether a message said nothing at all — no text, no parts, nothing to show a person. */
+function isSilent(message: Message): boolean {
+  const content = (message as { content?: unknown }).content;
+  if (content === undefined || content === null) return true;
+  if (typeof content === "string") return content.length === 0;
+  if (Array.isArray(content)) return content.length === 0;
+  return false;
+}
+
+/**
+ * Refuse to re-present a conversation the model API will reject.
+ *
+ * FOUND IN PRODUCTION, TWICE, ON BOTH PATHS. First on routines: two firings of one routine, fifteen
+ * minutes apart, both failed with `Tool result is missing for tool call
+ * call_TTbiXzJVNifQt8ioU1JJmj4S.` — the SAME call id both times, so it did not come from the live
+ * turn: the channel's Intelligence thread held an assistant message carrying a tool call whose
+ * result message never landed, because an earlier CHAT turn was interrupted mid-call. Then on chat
+ * itself: `AI_MissingToolResultsError: Tool result is missing for tool call
+ * chatcmpl-tool-8dd56dc7497c5ea9`, thrown three times in a row on one person's next three attempts
+ * to say anything. There the damaged message was not even durable: a frontend tool handler was torn
+ * down mid-run, so the live agent's messages in the browser held the call, the store did not, and
+ * every retry sent the same unanswerable call back up as `input.messages`.
+ *
+ * A model provider validates call/result pairing, so one historical dangle poisons EVERY later turn
+ * that replays it: on routines until the fatigue rule disables the routine, and on chat until the
+ * person works out for themselves that the conversation is dead and starts another one. A permanent
+ * failure grown out of transient damage, and nothing the person did wrong.
+ *
+ * WHY DROPPING IS THE RIGHT ANSWER, and not repair. History here is CONTEXT for a turn, not a
+ * transaction to resume. A dangling call is already permanently unanswerable — the tool run that
+ * would have answered it ended when that turn did, and there is no result to invent. The only two
+ * options are to seed a conversation the API refuses, or to seed the same conversation minus a call
+ * that never completed. The second one loses a fragment of an interrupted exchange; the first one
+ * takes the conversation away.
+ *
+ * WHAT THIS DOES NOT DO. It does not DELETE anything from the platform. The thread still holds every
+ * row and the person still sees the interrupted exchange in their channel. This is a read-side
+ * filter on one turn's input and nothing more.
+ *
+ * IDS ARE NEVER CHANGED, which is what keeps `persistedInputMessages`' id-subtraction in
+ * `run-turn.ts` correct: a message this pass stripped a tool call from keeps its id and is still
+ * subtracted out as historic, and a message it dropped was never a candidate to persist. So
+ * sanitizing cannot turn a firing into one that re-persists the transcript.
+ *
+ * The rules, in order:
+ *  1. A tool call is ANSWERED if some later message carries it as `toolCallId`, or if the caller
+ *     says it is answered elsewhere. Later, not merely present: a result ahead of its call is not a
+ *     pairing any provider accepts either.
+ *  2. An assistant message keeps only its answered calls. If that leaves it with no calls and
+ *     nothing said, the message is dropped — an empty assistant husk is itself invalid for some
+ *     providers, so stripping the call is not enough.
+ *  3. A tool result whose `toolCallId` matches no surviving call is dropped: the mirror-image dangle,
+ *     which is what an interruption between the two rows leaves behind in the other order.
+ *
+ * Order is preserved, the input array is not mutated, and a message the pass does not change is
+ * returned as the same object — a healthy thread, which is nearly all of them, goes through
+ * untouched rather than through a re-normalization that could quietly differ.
+ *
+ * @param answeredElsewhere Call ids that are about to be answered by something this history cannot
+ * see, and so must survive. That is the interrupt resume: `BuiltInAgent.run` appends a tool result
+ * per `input.resume` entry, keyed by `interruptId`, AFTER converting the messages
+ * (`@copilotkit/runtime/dist/agent/index.mjs`, the `resumeEntries` block in `run`). Dropping the
+ * call that the resume answers would turn a resumable interrupt into an orphaned result, which is
+ * the same error seen from the other side.
+ */
+export function sanitizeSeededHistory(
+  history: Message[],
+  answeredElsewhere: ReadonlySet<string> = new Set(),
+): Message[] {
+  /** Every position that answers a call id, in order. */
+  const answersFor = new Map<string, number[]>();
+  for (const [index, message] of history.entries()) {
+    const { toolCallId } = message as { toolCallId?: string };
+    if (toolCallId === undefined) continue;
+    answersFor.set(toolCallId, [...(answersFor.get(toolCallId) ?? []), index]);
+  }
+  /*
+   * Where a call's answer may still land: before the next thing a person or the system said.
+   *
+   * The model API walks the conversation in order and refuses it the moment a user or system
+   * message arrives while a call is still unanswered. So a result that turns up after a later
+   * user message does not answer anything, however real it was. This happened live: a browser
+   * tool handler resolved late, its result was appended after the person had already typed the
+   * next message, and the call read as answered here while the API still threw on every retry.
+   */
+  const boundaryAfter: number[] = new Array(history.length).fill(
+    history.length,
+  );
+  for (
+    let index = history.length - 1, next = history.length;
+    index >= 0;
+    index -= 1
+  ) {
+    boundaryAfter[index] = next;
+    const { role } = history[index] as { role?: string };
+    if (role === "user" || role === "system" || role === "developer")
+      next = index;
+  }
+  const answeredWithin = (id: string, index: number): boolean =>
+    (answersFor.get(id) ?? []).some(
+      (at) => at > index && at < (boundaryAfter[index] ?? history.length),
+    );
+
+  const surviving = new Set<string>();
+  const kept: (Message | undefined)[] = history.map((message, index) => {
+    const { toolCalls } = message as { toolCalls?: ToolCall[] };
+    if (toolCalls === undefined) return message;
+
+    const answered = toolCalls.filter((call) => {
+      if (answeredElsewhere.has(call.id)) return true;
+      return answeredWithin(call.id, index);
+    });
+    for (const call of answered) surviving.add(call.id);
+
+    // The husk check goes FIRST so it also catches a row that arrived with no calls and nothing
+    // said — the same invalid shape, reached without a dangle.
+    if (answered.length === 0 && isSilent(message)) return undefined;
+    // The healthy path, and the only one that returns the very same object.
+    if (answered.length === toolCalls.length) return message;
+
+    /*
+     * Cast for the same reason `toAgentMessage` casts: `Message` is a union discriminated on `role`,
+     * and a spread over the union widens past every branch of it. Neither rewrite here can change
+     * the role or the shape — one narrows the `toolCalls` array, the other removes the key — so
+     * there is nothing to narrow against and nothing that could stop being a `Message`.
+     */
+    if (answered.length > 0) {
+      return { ...message, toolCalls: answered } as Message;
+    }
+    // Text it did say, minus a call it cannot complete.
+    const { toolCalls: _dropped, ...rest } = message as Message & {
+      toolCalls?: ToolCall[];
+    };
+    return rest as Message;
+  });
+
+  return kept.filter((message): message is Message => {
+    if (message === undefined) return false;
+    const { toolCallId } = message as { toolCallId?: string };
+    return toolCallId === undefined || surviving.has(toolCallId);
+  });
+}

server/src/copilot.ts

@@ -14,6 +14,7 @@ import {
   COMPUTER_GUIDANCE,
   PROVENANCE_GUIDANCE,
 } from "../../shared/bot-prompt";
+import { sanitizeSeededHistory } from "./agents/history-sanitize";
 import type { AgentActor } from "./agents/profile-types";
 import type { AgentFetch, StallGuard } from "./channels/stall-guard";
 import type { DeploymentConfig } from "./config";
@@ -419,7 +420,7 @@ async function buildAgent(
    * what keeps a narrowed run from being told it holds something it was not offered.
    */
   const withTools = (tools: GrantedTool[]) =>
-    new BuiltInAgent(
+    new BuiltInAgentWithSaneHistory(
       builtInAgentConfiguration(
         agent,
         model,
@@ -666,6 +667,68 @@ function remoteAgentWithStandingRole(
   return remote;
 }
 
+/**
+ * A built-in Bot that will not hand the model provider a conversation it is going to refuse.
+ *
+ * FOUND LIVE, ON CHAT. One person's next three messages each failed with
+ * `AI_MissingToolResultsError: Tool result is missing for tool call chatcmpl-tool-8dd56dc7497c5ea9`,
+ * thrown out of the AI SDK's `convertToLanguageModelPrompt`. A frontend tool handler had been torn
+ * down while its call was open, so the agent's live messages in the browser carried an assistant
+ * message whose tool call never got a result. The durable store did not have it, nothing was going
+ * to answer it, and every retry sent it straight back up as `input.messages`. The conversation was
+ * finished until the person worked out for themselves to start another one.
+ *
+ * The guard has to be on this side of `run`. `BuiltInAgent.run` converts `input.messages` itself,
+ * with no seam in between, so wrapping the agent is the only place left to stand. The reasoning for
+ * why a dangling call is DROPPED rather than repaired, and why ids are never changed, is in
+ * `agents/history-sanitize.ts`, where the routines path found the same failure first.
+ *
+ * A RESUMED CALL IS NOT A DANGLE. `run` appends a tool result for each `input.resume` entry by
+ * `interruptId` AFTER converting the messages, so a call that a resume is about to answer must
+ * survive this pass or the appended result lands on nothing.
+ */
+class BuiltInAgentWithSaneHistory extends BuiltInAgent {
+  /**
+   * The configuration, held a second time because the base class keeps its own copy private and
+   * {@link clone} has to build another one of THIS class rather than of the base.
+   */
+  private readonly configuration: BuiltInAgentConfiguration;
+
+  constructor(configuration: BuiltInAgentConfiguration) {
+    super(configuration);
+    this.configuration = configuration;
+  }
+
+  run(input: RunAgentInput): Observable<BaseEvent> {
+    const answeredByResume = new Set(
+      (input.resume ?? []).map((entry) => entry.interruptId),
+    );
+    return super.run({
+      ...input,
+      messages: sanitizeSeededHistory(input.messages, answeredByResume),
+    });
+  }
+
+  /**
+   * Carried by hand, for the same reason {@link RunBuiltAgent.clone} is.
+   *
+   * The runtime clones an agent before every run, and the base class's clone hard-codes
+   * `new BuiltInAgent(this.config)`: inherited unchanged, the very first message anybody sends
+   * would go through an agent that does none of the above. The middleware list is copied because
+   * the base clone copies it, and it is reached through a cast because `AbstractAgent` declares it
+   * private. Nothing registers middleware on a built-in Bot today, and this is here so that the day
+   * something does, it is not lost in a clone.
+   */
+  clone(): BuiltInAgentWithSaneHistory {
+    const cloned = new BuiltInAgentWithSaneHistory(this.configuration);
+    type WithMiddlewares = { middlewares: unknown[] };
+    (cloned as unknown as WithMiddlewares).middlewares = [
+      ...(this as unknown as WithMiddlewares).middlewares,
+    ];
+    return cloned;
+  }
+}
+
 /**
  * An agent whose tools are decided when the run starts, because that is the first moment anybody
  * knows what the run is about, and who is asking on whose behalf.