fix(agent): stop shipping the model's scratchpad to the user - #1742
Merged
Conversation
Kris, on the first turn after the ask_user deploy: "kinda a mess". One reply,
1630 characters, that opened with three lines of debugging narration, stated
the same answer three times, and ran together mid-word:
Oops, let me fix that written file.Let me just avoid the script entirely
and use jq (per Rule 9a - avoid inline scripts anyway).Bad literal \n. Let
me repair it per Rule 9a.None of the 18 items found match...
Two independent defects, both pre-existing. Neither is the interruption bug
fixed in #1740 — that one stayed fixed: zero ask_user calls, zero
question.resolve lines, no claim that anyone interrupted anything.
1. THE GATEWAY'S FUSED FINAL OVERWRITES THE CORRECT ANSWER
Dev run a33a3f92 produced six assistant messages, five ending
stopReason=toolUse. resp_len was exactly their sum.
The harness already segments this correctly: `_accumulate_assistant` resets on
a tool boundary, so `agent_assistant_accum` holds the terminal block alone.
Then the chat channel's final arrives carrying every block fused into ONE
string with no separator, and
elif state == "final":
if text:
response_text = text
assigns it straight over the good value. `_extract_text` cannot help — its
"\n\n" join (#1138 F4) needs multiple blocks, and this arrives as one.
`_prefer_terminal_segment` undoes it, and is deliberately narrow: it fires
only when the chat text is a STRICT SUPERSET that ENDS WITH the accumulated
segment, which is what fusion looks like and what nothing else does. An empty
accumulator, an equal string, or a segment appearing mid-string is returned
untouched, so a gateway sending a clean — or genuinely richer — final still
wins.
Why two years of ReplyIsTheAnswerOnly replays never caught it: FakeGateway's
final is a bare `{"state": "final"}` with no message, so those tests never
reach that assignment at all. The new `chat_final()` helper carries text and
closes the blind spot. Verified both directions — with the fix neutralised, 3
of the 5 new behaviour tests fail and the 2 control cases still pass, so they
catch the bug without over-reaching.
2. RULE 1 TOLD THE MODEL SOMETHING FALSE
Rule 1 opened: "Reasoning, plans, tool calls, debugging steps, and
self-narration are internal scratchpad." They are not internal. Every text
block the model emits is delivered. We have spent a month adding forbidden
phrasings on top of a premise that told the model it had a private notepad —
of course it kept using one.
It now opens with what is actually true (YOU HAVE NO SCRATCHPAD; every line
you write is delivered), quotes the real fused reply, and carries the
measurement: across this workspace's 107 turns of history, 56% ship more than
one text block, and on those a median 51% of what the user reads is narration.
Worst case 98%.
The operative instruction changed with it. "Before sending, re-read the draft
and strike..." is meaningless for text emitted three tool calls ago, so the
check moved to before you TYPE: if what you are about to write is not the
finished answer, call the next tool instead.
Added "write the answer once" — the doubled answer in that reply was a
conclusion written before the psd-failure-report call and then restated after
it. The self-check line now asks whether this is the first and only prose
written this turn.
The two halves are complementary, not redundant: the code stops the fused
delivery, the rule stops the narration being written at all. Either alone
leaves half the symptom.
Gate: lint clean (zero warnings), typecheck clean, 5918 jest, 409 agent-image
python (+10 new), bootstrap budget 46914/80000, config-consistency and
eval-coverage gates pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kris, on the first turn after the #1740 deploy: "kinda a mess." One reply, 1630 characters, opening with three lines of debugging narration, stating the same answer three times, running together mid-word:
Two independent defects, both pre-existing. Neither is the interruption bug — that stayed fixed: zero
ask_usercalls, zeroquestion.resolvelines, no claim that anyone interrupted anything.1. The gateway's fused final overwrites the correct answer
Dev run
a33a3f92: six assistant messages, five endingstopReason=toolUse.resp_lenwas exactly their sum.The harness already segments this correctly —
_accumulate_assistantresets on a tool boundary, soagent_assistant_accumholds the terminal block alone. Then the chat channel's final arrives carrying every block fused into one string with no separator, and this assigns it straight over the good value:_extract_textcan't help: its\n\njoin (#1138 F4) needs multiple blocks, and this arrives as one.Reproduced against the real adapter before writing a line of fix, then again after:
_prefer_terminal_segmentis deliberately narrow. It fires only when the chat text is a strict superset that ends with the accumulated segment — what fusion looks like, and what nothing else does. Empty accumulator, equal string, or the segment appearing mid-string: untouched. A gateway sending a clean (or genuinely richer) final still wins.Why two years of
ReplyIsTheAnswerOnlyreplays never caught it:FakeGateway's final is a bare{"state": "final"}with no message, so those tests never reach that assignment at all. The newchat_final()helper carries text and closes the blind spot.Verified both directions — with the fix neutralised, 3 of the 5 new behaviour tests fail and both control cases still pass, so they catch the bug without over-reaching.
2. Rule 1 told the model something false
It opened: "Reasoning, plans, tool calls, debugging steps, and self-narration are internal scratchpad."
They are not internal. Every text block the model emits is delivered. We have spent a month adding forbidden phrasings on top of a premise telling the model it had a private notepad — of course it kept using one.
Rule 1 now opens with what is actually true, quotes the real fused reply, and carries the measurement. Across this workspace's 107 turns of history:
The operative instruction changed with it. "Before sending, re-read the draft and strike…" is meaningless for text emitted three tool calls ago, so the check moved to before you type: if what you're about to write isn't the finished answer, call the next tool instead.
Added write the answer once — the doubled answer was a conclusion written before the
psd-failure-reportcall and restated after it. The self-check line now asks whether this is the first and only prose written this turn.Why both halves
They are complementary, not redundant. The code stops the fused delivery; the rule stops the narration being written at all. Either alone leaves half the symptom: without the rule the agent still burns tokens narrating into the void, and without the code any future multi-block turn fuses again.
Verification
bun run lintclean (zero warnings) ·bun run typecheckclean · 5,918 jest · 409 agent-image python (+10 new) · bootstrap budget 46,914 / 80,000 ·check_config_consistency.pyandcheck_eval_coverage.pypass.Deploy
Agent image rebuild only — no web tier, no CDK.