Improve agent failure UX: surface structured errors (quota, auth, rate limit) instead of raw stderr tails

[gabrielvaz]

Summary

When an agent fails, Open Design renders a raw agent exited with code N message followed by the last 400 bytes of stderr. For most non-trivial failures this dumps an unparsed stack trace into the chat that is useless to non-developer users (and not great even for developers — the actual diagnostic message is often not in the last 400 bytes).

Repro (real example, today)

Triggered the Gemini CLI agent and hit Google's per-model capacity limit. The UI showed:

agent exited with code 1 nc file:///Users/gabriel.vaz/.npm-packages/lib/node_modules/@google/gemini-cli/bundle/gemini-U6FSFXFH.js:10811:26 at async main (file:///Users/gabriel.vaz/.npm-packages/lib/node_modules/@google/gemini-cli/bundle/gemini-U6FSFXFH.js:15885:5) {
  cause: {
    code: 429,
    message: 'You have exhausted your capacity on this model.',
    details: [ [Object], [Object] ]
  },
  retryDelayMs: 10000
}

The signal in there — code: 429, 'You have exhausted your capacity on this model.', retryDelayMs: 10000 — is exactly what the user needs to see, but it's buried in a JS stack trace because Open Design just slices the tail of stderr.

Source

apps/web/src/providers/daemon.ts:327-333

if (endStatus === 'failed' || exitSignal || (exitCode !== null && exitCode !== 0)) {
  const tail = stderrBuf.trim().slice(-400);
  handlers.onError(
    new Error(`agent exited with ${exitSignal ? `signal ${exitSignal}` : `code ${exitCode}`}${tail ? `\n${tail}` : ''}`),
  );
  return;
}

There are two problems:

1. The error is constructed as a single, lossy string. Anything structured the daemon already knows (which agent, which command, which model, what the exit code means for this CLI) is collapsed.
2. The 400-char stderr tail is a heuristic that frequently truncates the actual error message and keeps only the trailing stack frames — which is exactly what we see in the example above.

The daemon already has clear classification for HTTP errors elsewhere (server.ts:3987 maps 429 → RATE_LIMITED, etc.), but this same intelligence is not applied to agent CLI exits.

Requested change

1) Parse and classify common failure modes

Before falling back to the generic message, run stderr through a small set of per-agent matchers and surface a typed error to the UI:

Pattern (any agent)	UI category	Example user-facing message
code: 429, RESOURCE_EXHAUSTED, rate limit, capacity on this model	quota_exhausted	"Gemini hit a per-model capacity limit. Try again in ~10s, switch models, or use your own API key."
401, 403, Not logged in, auth, Please run /login	auth_required	"Claude Code needs to log in. Run /login in a terminal."
ENAMETOOLONG, prompt exceeds the safe size	prompt_too_large	"The composed prompt is too large for this CLI's argv limit."
Unknown arguments: ...	cli_version_mismatch	"Your installed Gemini CLI is too old (or too new) for the flags Open Design passes. See #978."
ENOENT, spawn ... ENOENT	binary_not_found	"Could not find claude on PATH."
Any other non-zero exit	agent_crashed	Show the full stderr in a collapsible panel, not a 400-char tail.

These categories are stable contracts the UI can branch on (icons, retry buttons, links to docs).

2) Don't truncate the diagnostic

Keep the full stderr buffer per run (it is already accumulated as stderrBuf). When showing it in chat, render it inside a collapsible "Show details" disclosure — short message on top, full output one click away. The 400-byte slice is hostile to debugging.

3) Honor retryDelayMs when present

Gemini CLI emits retryDelayMs for transient failures. When the classifier detects a retryable error (429, 500, 503), Open Design could:

• show a "Retry in 10s" countdown button in the failed message, and/or
• automatically retry once with backoff for transient categories (with a setting toggle).

4) Cross-link other failure-mode issues

Several existing bugs are symptoms of the same root cause (no structured error reporting):

• [Gemini CLI] DMG app picks up old gemini binary on PATH → Unknown arguments: output-format, outputFormat #978 — Unknown arguments: output-format, outputFormat (Gemini CLI version mismatch). Could be auto-classified.
• Problem YOLO in Gemini CLI #836 — YOLO not supported (Gemini CLI version mismatch).
• [Gemini CLI] agent exited with code 1 #736 — Gemini "agent exited with code 1" Node loader failure.
• Opencode on windows 11 - "agent exited with code 1 session not found" #677 — Opencode "session not found".
• AI Doesn't Receive Instructions #470 — "AI Doesn't Receive Instructions" — silent stderr.
• Error: Not logged in Please run /login #500 — Codex Not logged in / Please run /login.

A single error-classifier surface would let each of these become a one-click "fix this" UI hint instead of a raw trace.

Out of scope (but worth considering)

• A debug button that opens the full run log file (the daemon already writes one) so users don't have to copy-paste the stack trace into a bug report.
• An "Open in terminal" button that re-runs the same agent invocation in the user's shell so they can debug live.

Acceptance criteria

• A 429 from Gemini renders as a one-line user-facing message ("capacity exhausted, try again in 10s") with the full stack trace tucked under a disclosure.
• A missing-binary failure renders as "Could not find X — install instructions" instead of spawn ENOENT.
• The chat history no longer shows truncated mid-stack-frame strings.

[lefarcen]

Thanks for the detailed report @gabrielvaz — I checked the current chat-agent failure path and this is confirmed.

Today the web client accumulates stderr chunks during /api/runs/:id/events, but generic non-zero exits are converted into a plain Error.message with only stderrBuf.trim().slice(-400) appended (apps/web/src/providers/daemon.ts:331-335). The daemon does already have structured SSE errors for some paths, and there are HTTP classification patterns elsewhere (401 → UNAUTHORIZED, 429 → RATE_LIMITED, etc.), but generic child-process exits currently just finish the run as failed with code/signal and leave the web client to synthesize the final message.

So the root problem is real: most agent CLI failures don't have a typed failure contract, and the final UI message is lossy.

A good fix should probably live in the daemon rather than only the web client:

• Accumulate bounded/redacted stderr for the run in the daemon close path.
• Classify common agent failure modes before finish(run, 'failed', ...).
• Emit a structured chat SSE error with a contract-backed category, short message, full details, and optional retryDelayMs before the end.
• Web client then renders that error prominently with a disclosure panel for full details, and respects retry signals.

The web disclosure rendering and auto-retry countdown are easier follow-ups once the daemon classification contract is stable. The main design decision is: should categories like quota_exhausted, auth_required, cli_version_mismatch, binary_not_found be new top-level ApiErrorCode values, or a new AgentFailureCategory in typed details? The latter keeps the core contract broad (AGENT_EXECUTION_FAILED, AGENT_UNAVAILABLE, etc.) and lets the UI branch on richer subcategories.

The linked issues you mentioned (#978, #836, #736, #677, #500) would all benefit from this shared classifier — they're currently handled case-by-case or left as raw traces.

This is a solid direction. If you or someone else wants to take a stab at the daemon classification layer, I'd recommend:

1. Add tests with stub child processes for the main patterns (429/quota, auth required, prompt too large, unknown arguments, ENOENT, unclassified crash).
2. Wire classification in apps/daemon/src/server.ts right before the run finish, emitting error before end.
3. Update the web client to render that error gracefully.

Otherwise a maintainer will likely pick this up when prioritizing the next wave of agent UX improvements.

[lefarcen]

This is open for community help.

If you'd like to work on it, comment /claim below. The first /claim after the latest release gets it. Any comment, linked PR, or progress note keeps the claim active.

[michaelmanly]

From the error, this looks like a rate limit problem on gemini. Trying npx ai-doctor might help. It can fix weak retry strategy with exponential backoff, jitter, retry caps, and retry-after handling.

[bulai0408]

/claim

[lefarcen]

Claimed by @bulai0408. Thanks for picking this up! 🎉

Quick question to time check-ins right — roughly how long do you think this will take?

• A few hours / today
• 1-3 days
• ~ 1 week
• Longer / unsure

Reply with whatever fits. If you don't reply, I'll default to checking in every ~12h. Either way, any comment, linked PR, or progress note keeps the claim active.

[lefarcen]

Quick update: @bulai0408 opened #2009 to address this failure mode.

The PR targets the lossy agent exited with code N + stderr-tail path by adding typed handling for common local-agent failures (quota/rate limit, auth required, prompt too large, CLI flag/version mismatch, missing binary, and network failures), while keeping the full stderr behind a details disclosure. We'll keep this issue open until the PR lands and the failed-run UX can be verified against the original Gemini 429 / retryDelayMs case.

[lefarcen]

Hi @bulai0408, linked PR #2009 has had no author activity for 24 hours. Please push or comment when you are ready.

[lefarcen]

Quick update: #3083 is also open against this failure-mode area alongside #2009.

#3083 tackles the auth/quota/upstream slice by classifying non-AMR model-service failures in the daemon and threading the structured code into chat, then showing a hosted-AMR nudge for those specific errors. #2009 remains the broader claimed PR for the stderr-classification/details-disclosure path, so we'll keep this issue open while maintainers compare the scopes and decide how the two efforts should land.

[lefarcen]

Quick partial-shipped update for @gabrielvaz and @michaelmanly: #3083 has merged on the AMR runtime branch with the auth/quota/upstream slice of this issue.

That change classifies non-AMR model-service auth, rate-limit/quota, and upstream failures into structured error codes, carries the code through chat status events, and lets the web UI show AMR-specific retry / authorize / top-up guidance where applicable.

Keeping this issue open because the broader #982 scope is not fully done yet: the general stderr/details-disclosure work — full diagnostics behind “Show details”, plus cases like missing binary, CLI version mismatch, prompt-too-large, and generic crashes — is still tracked by @bulai0408’s #2009.

If you hit the original Gemini 429 / retryDelayMs style case again once this AMR branch is in your build, please drop what you see here. The useful signal is whether the auth/quota path now gives actionable guidance instead of only a raw stderr tail.