tracking: Claude provider observer overflows context window — no bound on history size, zero memory saved on long sessions

[thedotmack]

Root Cause

The Claude provider's SDKAgent accumulates every observation into a single ever-growing conversation with no sliding window, token cap, or message limit. When long sessions with large tool outputs exceed the model's context window, the SDK returns 'Prompt is too long', the session aborts, and zero observations are saved. The OpenRouter provider has a truncateHistory mechanism; SDKAgent never received equivalent protection. A related sub-issue is that base64 image blobs in tool payloads bloat the prompt further without being stripped.

Tracked Items

Issues

• Closes Claude provider (SDKAgent) has no context bound — observer overflows with "Prompt is too long" and saves zero memory on long sessions #2956 — Claude provider (SDKAgent) has no context bound — observer overflows with "Prompt is too long" and saves zero memory on long sessions #2956
• Closes Observer silently drops observations on "Prompt is too long" — strip image/base64 payloads before sending to the model #2866 — Observer silently drops observations on "Prompt is too long" — strip image/base64 payloads before sending to the model #2866

PRs

• Related PR fix(sdk): bound Claude provider observer context to prevent overflow (#2956) #2957 — fix(sdk): bound Claude provider observer context to prevent overflow (#2956) #2957
• Related PR fix(observer): strip base64 image blobs from tool payloads before building observer prompt (closes #2866) #2884 — fix(observer): strip base64 image blobs from tool payloads before building observer prompt (closes #2866) #2884

Resolution

This is the canonical tracking issue for this root cause. Individual issues and PRs listed above will be closed in favor of this one.

[brandonjveldman]

Still live in 13.10.2, with a new failure mode: the overflow now arrives as prose and bypasses fresh-session recovery, so the observer gets stuck in an infinite "Prompt is too long" loop.

Commenting on this canonical tracker instead of opening a duplicate. The symptom you track here (observer overflows the context window, "Prompt is too long", zero memory saved) still reproduces after the server rename refactor, but the mechanism has shifted and the older throw-based recovery no longer fires. Adding the root cause specific to the current architecture plus a small recovery patch that complements the context-bounding work in #2957.

Environment

• Installed version: 13.10.2 (Claude Code plugin install, macOS).
• Provider: claude (CLAUDE_MEM_PROVIDER=claude), Sonnet-class observer model via CLAUDE_MEM_MODEL, roughly 200k context window.
• Data/logs under ~/.claude-mem.

Symptom

Long primary sessions stop producing observations entirely partway through. The observer hits "Prompt is too long", retries about once per second, and stores zero observations for the rest of the session. This is not a clean single abort. It is a continuous resume loop against the same poisoned conversation.

Root cause (the part not yet captured here)

The observer is one persistent SDK conversation per session, and on respawn it resumes the same memory session. When the cumulative context finally overflows, the SDK does not throw. It returns the literal assistant text "Prompt is too long".

The recovery machinery that clears memorySessionId and forces a fresh session only runs on a thrown error (src/services/worker/ClaudeProvider.ts, the catch path around lines 101-108). Because the overflow is a normal-looking assistant text response, no throw ever happens.

Meanwhile the output classifier has a dedicated detector for quota-limit prose (isQuotaLimitedObserverOutput in src/sdk/output-classifier.ts, around lines 57-72) but no equivalent for overflow prose. So the message falls through to the generic prose branch in src/services/worker/agents/ResponseProcessor.ts (around lines 74-89), which drops the queued batch and returns WITHOUT clearing memorySessionId or setting forceInit. The poisoned session is resumed on the next tick, overflows again, and the loop never exits for the remainder of the session.

The server refactor appears to have reintroduced the gap that the SDKAgent.ts era closed. The changelog entry "observer: drop invalid prose and pause on quota" (13.9.1) shows quota prose got a handler and generic prose gets dropped, but overflow prose was never given the reset that the old throw path performed. A quick check of the default branch is consistent with this: a code search for isQuotaLimitedObserverOutput returns matches, while a search for any isContextOverflow-style detector returns none.

Log signature

Identical in every storm:

• assistant text "Prompt is too long" classified as prose (not an error, no throw)
• promptNumber pinned at 1 (the resumed session never advances)
• [PARSER] SDK returned non-XML prose response, ignoring queued batch
• zero observation XML stored

Observed magnitude

In local logs, one session logged 633 consecutive overflow events at a queued-buffer depth of 149 items. A separate day logged 246 overflow events across two sessions. Both volume-driven sessions (heavy Bash and Read traffic) and image-driven sessions (many screenshot payloads) reach the same terminal loop.

Relationship to existing tracked work

• fix(sdk): bound Claude provider observer context to prevent overflow (#2956) #2957 bounds the observer context to prevent the overflow. That is prevention. The patch below is recovery, and the two are complementary: even with a bound in place, an overflow that slips through should reset rather than loop, and even without the recovery, bounding reduces how often the loop is reached.
• Claude provider (SDKAgent) has no context bound — observer overflows with "Prompt is too long" and saves zero memory on long sessions #2956 assumed the overflow throws and aborts. On 13.10.2 that is no longer true, which is why its cited recovery does not engage.
• Bug: "Prompt is too long" response causes infinite CPU loop instead of graceful termination #870 (the original infinite CPU loop on "Prompt is too long") is effectively regressing through the new prose path.

Proposed patch (two files, mirrors the existing quota-prose handling)

File 1: src/sdk/output-classifier.ts, add alongside isQuotaLimitedObserverOutput:

export function isContextOverflowObserverOutput(raw: unknown): boolean {
  if (typeof raw !== 'string' || raw.trim() === '') return false;
  const text = raw.toLowerCase().replace(/\s+/g, ' ').trim();
  return /\bprompt is too long\b/.test(text)
      || /\binput is too long\b/.test(text)
      || /\bcontext (window|length)\b.*\b(exceed|exceeded|too long)\b/.test(text);
}

File 2: src/services/worker/agents/ResponseProcessor.ts, insert immediately after the quota branch, before the generic prose branch:

if (isContextOverflowObserverOutput(text)) {
  session.consecutiveInvalidOutputs = 0;
  logger.warn('PARSER', `${agentName} hit context overflow (prose): resetting to a fresh observer session`, {
    sessionId: session.sessionDbId, outputClass: 'prose', preview: previewOutput(text),
  });
  session.memorySessionId = null;   // stop resuming the overflowed transcript
  session.forceInit = true;         // next spawn starts fresh
  await sessionManager.resetProcessingToPending(session.sessionDbId); // preserve the batch
  try { session.abortController.abort(); } catch {}
  worker?.broadcastProcessingStatus?.();
  return;
}

This reuses the machinery the quota path already relies on (resetProcessingToPending, forceInit, the resume gate). The effect is to convert an infinite zero-yield storm into a one-time fresh-session reset that then drains the preserved backlog.

Verification status

The classifier gap and the two insertion points are verified against the installed 13.10.2 bundle (quota detector present, overflow detector absent, generic prose branch returns without a reset). The end-to-end recovery behavior of the patch is reasoned from the symmetric quota-recovery path rather than runtime-verified, since a bundle rebuild is required to exercise it.

Happy to open a PR with this if it is useful. Let me know whether you would prefer it as a standalone recovery fix or folded in alongside the bounding work in #2957.