Skip to content

fix(agent): stop shipping the model's scratchpad to the user - #1742

Merged
krishagel merged 1 commit into
devfrom
fix/agent-scratchpad-leak
Sep 6, 2026
Merged

fix(agent): stop shipping the model's scratchpad to the user#1742
krishagel merged 1 commit into
devfrom
fix/agent-scratchpad-leak

Conversation

@krishagel

Copy link
Copy Markdown
Member

Kris, on the first turn after the #1740 deploy: "kinda a mess." One reply, 1630 characters, opening with three lines of debugging narration, stating the same answer three times, running together mid-word:

Oops, let me fix that written file.Let me just avoid the script entirely and
use jq (per Rule 9a - avoid inline scripts anyway).Bad literal \n. Let me
repair it per Rule 9a.None of the 18 items found match...

Two independent defects, both pre-existing. Neither is the interruption bug — that stayed fixed: zero ask_user calls, zero question.resolve lines, no claim that anyone interrupted anything.

1. The gateway's fused final overwrites the correct answer

Dev run a33a3f92: six assistant messages, five ending stopReason=toolUse. resp_len was exactly their sum.

The harness already segments this correctly_accumulate_assistant resets on a tool boundary, so agent_assistant_accum holds the terminal block alone. Then the chat channel's final arrives carrying every block fused into one string with no separator, and this assigns it straight over the good value:

elif state == "final":
    if text:
        response_text = text

_extract_text can't help: its \n\n join (#1138 F4) needs multiple blocks, and this arrives as one.

Reproduced against the real adapter before writing a line of fix, then again after:

before:  delivered 229 chars (fused 229, terminal 57)  -> REPRODUCED
after:   delivered  57 chars                           -> terminal segment only

_prefer_terminal_segment is deliberately narrow. It fires only when the chat text is a strict superset that ends with the accumulated segment — what fusion looks like, and what nothing else does. Empty accumulator, equal string, or the segment appearing mid-string: untouched. A gateway sending a clean (or genuinely richer) final still wins.

Why two years of ReplyIsTheAnswerOnly replays never caught it: FakeGateway's final is a bare {"state": "final"} with no message, so those tests never reach that assignment at all. The new chat_final() helper carries text and closes the blind spot.

Verified both directions — with the fix neutralised, 3 of the 5 new behaviour tests fail and both control cases still pass, so they catch the bug without over-reaching.

2. Rule 1 told the model something false

It opened: "Reasoning, plans, tool calls, debugging steps, and self-narration are internal scratchpad."

They are not internal. Every text block the model emits is delivered. We have spent a month adding forbidden phrasings on top of a premise telling the model it had a private notepad — of course it kept using one.

Rule 1 now opens with what is actually true, quotes the real fused reply, and carries the measurement. Across this workspace's 107 turns of history:

  • 56% ship more than one text block
  • on those, a median 51% of what the user reads is narration
  • worst case 98%

The operative instruction changed with it. "Before sending, re-read the draft and strike…" is meaningless for text emitted three tool calls ago, so the check moved to before you type: if what you're about to write isn't the finished answer, call the next tool instead.

Added write the answer once — the doubled answer was a conclusion written before the psd-failure-report call and restated after it. The self-check line now asks whether this is the first and only prose written this turn.

Why both halves

They are complementary, not redundant. The code stops the fused delivery; the rule stops the narration being written at all. Either alone leaves half the symptom: without the rule the agent still burns tokens narrating into the void, and without the code any future multi-block turn fuses again.

Verification

bun run lint clean (zero warnings) · bun run typecheck clean · 5,918 jest · 409 agent-image python (+10 new) · bootstrap budget 46,914 / 80,000 · check_config_consistency.py and check_eval_coverage.py pass.

Deploy

Agent image rebuild only — no web tier, no CDK.

Kris, on the first turn after the ask_user deploy: "kinda a mess". One reply,
1630 characters, that opened with three lines of debugging narration, stated
the same answer three times, and ran together mid-word:

    Oops, let me fix that written file.Let me just avoid the script entirely
    and use jq (per Rule 9a - avoid inline scripts anyway).Bad literal \n. Let
    me repair it per Rule 9a.None of the 18 items found match...

Two independent defects, both pre-existing. Neither is the interruption bug
fixed in #1740 — that one stayed fixed: zero ask_user calls, zero
question.resolve lines, no claim that anyone interrupted anything.

1. THE GATEWAY'S FUSED FINAL OVERWRITES THE CORRECT ANSWER

Dev run a33a3f92 produced six assistant messages, five ending
stopReason=toolUse. resp_len was exactly their sum.

The harness already segments this correctly: `_accumulate_assistant` resets on
a tool boundary, so `agent_assistant_accum` holds the terminal block alone.
Then the chat channel's final arrives carrying every block fused into ONE
string with no separator, and

    elif state == "final":
        if text:
            response_text = text

assigns it straight over the good value. `_extract_text` cannot help — its
"\n\n" join (#1138 F4) needs multiple blocks, and this arrives as one.

`_prefer_terminal_segment` undoes it, and is deliberately narrow: it fires
only when the chat text is a STRICT SUPERSET that ENDS WITH the accumulated
segment, which is what fusion looks like and what nothing else does. An empty
accumulator, an equal string, or a segment appearing mid-string is returned
untouched, so a gateway sending a clean — or genuinely richer — final still
wins.

Why two years of ReplyIsTheAnswerOnly replays never caught it: FakeGateway's
final is a bare `{"state": "final"}` with no message, so those tests never
reach that assignment at all. The new `chat_final()` helper carries text and
closes the blind spot. Verified both directions — with the fix neutralised, 3
of the 5 new behaviour tests fail and the 2 control cases still pass, so they
catch the bug without over-reaching.

2. RULE 1 TOLD THE MODEL SOMETHING FALSE

Rule 1 opened: "Reasoning, plans, tool calls, debugging steps, and
self-narration are internal scratchpad." They are not internal. Every text
block the model emits is delivered. We have spent a month adding forbidden
phrasings on top of a premise that told the model it had a private notepad —
of course it kept using one.

It now opens with what is actually true (YOU HAVE NO SCRATCHPAD; every line
you write is delivered), quotes the real fused reply, and carries the
measurement: across this workspace's 107 turns of history, 56% ship more than
one text block, and on those a median 51% of what the user reads is narration.
Worst case 98%.

The operative instruction changed with it. "Before sending, re-read the draft
and strike..." is meaningless for text emitted three tool calls ago, so the
check moved to before you TYPE: if what you are about to write is not the
finished answer, call the next tool instead.

Added "write the answer once" — the doubled answer in that reply was a
conclusion written before the psd-failure-report call and then restated after
it. The self-check line now asks whether this is the first and only prose
written this turn.

The two halves are complementary, not redundant: the code stops the fused
delivery, the rule stops the narration being written at all. Either alone
leaves half the symptom.

Gate: lint clean (zero warnings), typecheck clean, 5918 jest, 409 agent-image
python (+10 new), bootstrap budget 46914/80000, config-consistency and
eval-coverage gates pass.
Copilot AI lite review requested due to automatic review settings September 6, 2026 18:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@krishagel
krishagel merged commit 59024d2 into dev Sep 6, 2026
7 of 8 checks passed
@krishagel
krishagel deleted the fix/agent-scratchpad-leak branch September 6, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants