fix(temporal): raise Glitter synthesis token cap and retry on truncation - #1995
fix(temporal): raise Glitter synthesis token cap and retry on truncation#1995shepherdjerred wants to merge 2 commits into
Conversation
A pinned Glitter generation now clears all extraction (the sanitize fix works) but dies in synthesis with LengthFinishReasonError: gpt-5.6-sol runs at reasoning_effort "medium", so its hidden reasoning tokens plus the large style synthesis output crossed the 15k max_completion_tokens cap (LLM traces show 80-86k input / 4.7-7.6k output). The same snapshot+code passed on 2026-07-29, so this is sol model drift since the cache was populated. Raise SYNTHESIS_MAX_OUTPUT_TOKENS 15k->28k (headroom over the observed ~15k) and retry a synthesis call once at a 40k ceiling on LengthFinishReasonError. The preflight estimator and budget authorization use the 40k ceiling as the honest worst case. Adds a test that a synthesis call truncating at 28k succeeds on the 40k retry and the run completes. Probe: if synthesis then converges within finalizeStyleSynthesis, Phase 1 completes; the traces also show synthesis-repair loops (output failing the exact 20/30/18 contract), which would confirm a deeper sol re-tune is needed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WXxxmDmqcQas3cYqqE2nuU
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2fe4566eca
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const completion = await (async () => { | ||
| try { | ||
| return await parseGlitterCompletion(callSite, params); | ||
| } catch (error: unknown) { | ||
| if (!(error instanceof LengthFinishReasonError)) { | ||
| throw error; | ||
| } | ||
| return await parseGlitterCompletion(callSite, { | ||
| ...params, | ||
| max_completion_tokens: SYNTHESIS_TRUNCATION_RETRY_MAX_OUTPUT_TOKENS, |
There was a problem hiding this comment.
When the 28k request ends with LengthFinishReasonError, that completed API request has already consumed billable input/reasoning/output tokens, but the catch immediately issues the 40k request without persisting or recording the first request's usage. If the retry succeeds, the artifact and GenerationBudget receive only the retry's usage; if it also fails, no spend receipt is created and the workflow's second activity attempt can pay for both calls again. The preflight estimate and authorization likewise reserve only one 40k call rather than the 28k call plus its retry, so the advertised hard run budget and reported actual spend can be exceeded precisely in the truncation scenario this change handles.
Useful? React with 👍 / 👎.
| - 2026-08-04 (truncation fix): `fix/glitter-synthesis-truncation` raises | ||
| `SYNTHESIS_MAX_OUTPUT_TOKENS` 15k→28k (comfortable headroom over the observed | ||
| ~15k) and retries a synthesis call **once** at a 40k ceiling on | ||
| `LengthFinishReasonError` (imported from `openai/core/error`). The preflight | ||
| estimator + budget authorize now use the 40k ceiling as the honest worst case. | ||
| Test: a synthesis call that truncates at 28k succeeds on the 40k retry and the |
There was a problem hiding this comment.
This adds the 2026-08-04 implementation work to the plan, but the document still proceeds directly into the old ## Session Log — 2026-07-29 section and never records an August Done / Remaining / Caveats summary. Append the required 2026-08-04 session log so the unfinished probe and handoff state are captured in the repository's mandated format.
AGENTS.md reference: AGENTS.md:L125-L142
Useful? React with 👍 / 👎.
PR #1992 shipped @shepherdjerred/pr-fleet-web (packages/pr-fleet-controller/packages/web) without adding it to scripts/ci-test-manifest.json, so the `CI reporting manifest` test (scripts/ci-reporting.test.ts) fails and `bun run verify` is red on main. Add the workspace to the manifest and wire its `test:ci`/`test:report` scripts to the shared runner (matching the sibling pr-fleet-controller). Its tests already pass once the workspace is linked. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WXxxmDmqcQas3cYqqE2nuU
|
Added a second commit ( |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2dcdf15053
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return await parseGlitterCompletion(callSite, { | ||
| ...params, | ||
| max_completion_tokens: SYNTHESIS_TRUNCATION_RETRY_MAX_OUTPUT_TOKENS, | ||
| }); |
There was a problem hiding this comment.
When the 28k call truncates, this stores the 40k completion under a request hash that declares only maxCompletionTokens: 28_000. Consequently the reported artifact identity does not describe the request that produced it, and changing the retry ceiling later will continue reusing this stale artifact because the cache key will remain unchanged. Include the fallback policy/ceiling in the hashed request, or cache the retry as a distinct request.
Useful? React with 👍 / 👎.
1 participant
Why
With the extraction fixes (#1982 repair-loop, #1988 sanitize) deployed, a pinned
Glitter generation clears all extraction cleanly (zero citation errors) and
now dies in synthesis with
LengthFinishReasonError(finish_reason: length→ unparseable structured output).
LLM traces (
llm-archive) confirm the cause:gpt-5.6-solsynthesis runs80–86k input tokens, 4.7–7.6k output, at
reasoning_effort: "medium". Itshidden reasoning tokens + output share the 15k
max_completion_tokenscap,and one attempt crossed it. The same pinned snapshot + code succeeded on
2026-07-29, so this is
solmodel drift since the cache was populated.What
SYNTHESIS_MAX_OUTPUT_TOKENS15k → 28k (comfortable headroom over theobserved ~15k reasoning+output).
LengthFinishReasonError(imported from
openai/core/error).budget.authorizeUncachedCalluse the 40k ceilingas the honest worst-case output bound.
run completes.
Probe note
This is deliberately a probe (per plan
2026-07-29_glitter-style-card-v2.md).The traces also show
glitter-style-synthesis-repaircalls — synthesis output isalso failing
finalizeStyleSynthesis(the exact 20-quote/30-sample/18-examplecontract) and going into repair loops. If, after this fix, synthesis converges
within finalize validation, Phase 1 completes. If it still fails the exact-count
contract, that confirms a deeper
solre-tune is needed (contract counts can't besanitized by dropping, unlike extraction evidence).
Testing
workflow-bundle smoke all pass.
🤖 Generated with Claude Code