Silent-overflow recovery fails completed submission by continuing from assistant

[jgwesterlund]

Describe the Bug

With @flue/runtime 2.0.3, a durable submission can settle as failed after it has already produced a complete assistant response and threshold compaction succeeds.

The durable submission error is:

Cannot continue from message role: assistant

Observed event order:

1. Several model/tool cycles complete successfully.
2. A final assistant message completes with stopReason: "stop".
3. Threshold compaction emits a canonical compaction record parented to that assistant message.
4. A resource snapshot is emitted.
5. 57 ms later, the submission settles as failed with the error above.

There was no tool or model failure. The completed response used about 38.7k total model tokens. This happened twice in adjacent production submissions.

The agent declares useAgentStart and useResponseFinish, but no useAgentFinish, so no finish-hook continuation was expected.

Possible investigation points in v2.0.3:

• runModelTurnWithRecovery() performs threshold compaction and then returns.
• runWouldStopPhase() continuation paths call agentLoop.continue().
• continueRebuilt() only avoids continuing from an assistant leaf when a restart callback was supplied.

I do not have the internal stack trace, so the exact continuation entry point remains unconfirmed.

Expected Behavior

After a successful assistant response and successful threshold compaction, the submission should settle as completed and retain the assistant response. No further model turn should start unless new user/signal/join input exists.

Steps to Reproduce

1. Use @flue/runtime 2.0.3 with durable Postgres submissions and threshold compaction enabled.
2. Run a multi-tool conversation until a successful assistant turn crosses the compaction threshold.
3. Let threshold compaction complete after the assistant's stop response.
4. Inspect the durable submission settlement.

Observed result: compaction completes, then the submission fails with Cannot continue from message role: assistant.

Environment:

• @flue/runtime: 2.0.3
• @flue/postgres: 2.0.3
• @earendil-works/pi-agent-core: 0.83.0
• Node.js: 24.x

[jgwesterlund]

I found a deterministic full durable-runtime reproduction and the exact failing branch.

This is not threshold compaction. In the observed case, automatic threshold compaction was disabled. The model metadata declares contextWindow: 32_768, while the successful final assistant response reported:

{
  "stopReason": "stop",
  "usage": {
    "input": 2146,
    "cacheRead": 36352,
    "output": 176,
    "totalTokens": 38674
  }
}

pi-ai@0.83.0 treats a successful stop as silent overflow when usage.input + usage.cacheRead > contextWindow. Flue therefore enters the overflow-recovery branch even with compaction: false (which only disables threshold compaction):

• isAssistantContextOverflow() delegates to pi-ai's classifier
• runModelTurnWithRecovery() compacts, rebuilds canonical context, then sets start = continueRebuilt
• after compaction, rebuilt context still ends in the successful assistant; continueRebuilt() falls through to agentLoop.continue() because no restart callback exists
• pi-agent-core rejects that with Cannot continue from message role: assistant

I reproduced this through the full Node durable path using SQLite (assembleNodeAgentRuntime → dispatch → durable settlement), not only a direct session call. Deterministic setup:

1. Faux model: contextWindow: 32_768, maxTokens: 4_096.
2. Agent declares useModel(model, { compaction: false }) and no useAgentFinish.
3. First 70k-character dispatch creates enough retained context.
4. Second 70k-character dispatch returns a successful stop response.
5. pi-ai reports input+cacheRead above 32,768.
6. One summarization response runs, compaction commits, then submission settles failed with exact error.

Event/record order matches production: successful assistant_message_completed(stop) → compaction → submission_settled(failed).

A one-line guard immediately after successful overflow compaction makes the durable regression pass:

if (assistant.stopReason === 'stop') return;

That preserves compaction and the completed answer but skips impossible continuation from an assistant leaf. However, this changes intended silent-overflow behavior from “compact and retry” to “compact and settle”; if retrying a successful silent-overflow response is required, Flue needs to append/steer a canonical continuation signal (or restart with an explicit prompt) before calling continue(). Error-based overflow (stopReason: "error") must keep current compact-and-retry behavior.

[jgwesterlund]

Additional control test: the bug is specific to completed-assistant overflow shapes, not error-based overflow generally.

Using the same faux provider and compaction setup:

• stop + input/cacheRead above contextWindow → compaction succeeds, then exact Cannot continue from message role: assistant.
• length + zero output + input/cacheRead filling the window → same failure.
• error + explicit provider overflow message → compaction and continued recovery succeed (recovered answer).

So the fix should preserve current error-overflow compact-and-retry behavior. The broken invariant is attempting to retry a completed assistant (stop/classified length) without first placing canonical user/signal input after that assistant.

[jgwesterlund]

Confirmed application-side trigger provenance: the production revision intentionally supplies conservative custom-gateway metadata (contextWindow: 32_768) because the gateway has no trusted capability catalog. The upstream model accepted and completed a context whose reported input+cacheRead was ~35.7k / ~38.5k, so pi-ai's generic silent-overflow heuristic fired against metadata rather than an actual provider failure.

This means stale/undersized custom model metadata is a realistic trigger, not only a synthetic faux-model case. Flue still should not call continue() from a completed assistant leaf, but accurate per-model context-window metadata will also reduce false silent-overflow classification at the application boundary.