Skip to content

feat(daemon): expose per-run failure diagnostics β€” od run inspect + endpoint (slice 1 of #5489) - #5511

Draft
tomsen02 wants to merge 1 commit into
nexu-io:mainfrom
tomsen02:feat/run-diagnostics-slice1
Draft

feat(daemon): expose per-run failure diagnostics β€” od run inspect + endpoint (slice 1 of #5489)#5511
tomsen02 wants to merge 1 commit into
nexu-io:mainfrom
tomsen02:feat/run-diagnostics-slice1

Conversation

@tomsen02

Copy link
Copy Markdown
Contributor

Refs #5489. Draft β€” a concrete reference implementation of the slice-1 shape @lefarcen and I converged on in the proposal, posted to make the product-direction review easier. Not asking to merge ahead of that direction.

Why

Per #5489: the daemon already computes the full failure classification at finalize-time (for retry-policy + telemetry), but the API surface only carried failureCategory / failureDetail. failureStage, retryable, and user_action were telemetry-only β€” so a self-hoster, or an external agent driving Open Design through od, could not see why a run failed, whether it is safe to retry, or what action to take without reading PostHog/Langfuse. That breaks the UI+CLI dual-track rule and the CLI-as-embeddability-contract principle in AGENTS.md.

What this slice does (backbone: contracts + daemon + CLI + test)

  • Carries failureStage / failureRetryable / failureUserAction on the run (same source object as the existing category/detail, server.ts).
  • GET /api/runs/:id/diagnostics β€” a dedicated sub-resource so GET /api/runs/:id stays lean; returns RunDiagnosticsResponse with the full classification (failure is null for a run that did not fail).
  • od run inspect <runId> [--json].
  • Contracts: RunDiagnosticsResponse + re-export RunFailureStage / RunFailureUserAction (mirrors the existing RunFailureCategory / RunFailureDetail re-exports).
$ od run inspect run_abc
run_abc β€” failed
failure: auth / invalid_api_key   (stage: first_token_wait)
retryable: no
suggested action: login

The 2/5 β†’ 5/5 split (per the proposal thread)

This carries the bounded failure classification only. Availability matches GET /api/runs/:id (the live run registry), so it sidesteps the persistence seam entirely. The heavier timing-segment + token/cache breakdown β€” which is fire-and-forget into telemetry today and needs the storage decision (persist to SQLite under the RUNTIME_DATA_DIR contract vs in-memory only) β€” is deliberately not in this slice.

Deliberately deferred (call these out for review)

  • Web diagnostics panel (+ its i18n across 18 locales): held until the shape is blessed, so the panel and the timing/token fields land together rather than churning the UI twice. The dual-track intent is met by od run inspect here + the panel in the follow-up.
  • Durable / timing / token slice: separate PR, gated on the storage decision.

Scope boundaries

Pure exposure β€” no new computation, no change to classification logic (stays out of the accuracy-tuning lane #4966 / #5223 / #5356). Complementary to #5321 (which surfaces category/detail as chat guidance), not a duplicate.

Validation

  • pnpm --filter @open-design/daemon typecheck (both tsconfigs) β€” clean
  • pnpm guard β€” 78/78
  • apps/daemon/tests/run-diagnostics-endpoint.test.ts β€” green 3/3: fails a run through a fake claude over the production HTTP API, asserts /diagnostics now carries stage / retryable / userAction (control: the plain status endpoint does not).

Happy to adjust the endpoint shape, field names, or the persistence/UI phasing once the product direction lands.

Surface area

  • CLI β€” od run inspect
  • API / contract β€” GET /api/runs/:id/diagnostics + RunDiagnosticsResponse
  • UI β€” deferred to the follow-up (see above)

…dpoint)

Slice 1 of nexu-io#5489. The daemon already computes the full failure
classification at finalize-time for retry-policy + telemetry, but the
API surface only carried failureCategory / failureDetail β€” failureStage,
retryable, and user_action were telemetry-only, so a local user or an
external agent driving Open Design through `od` could not see why a run
failed, whether it is safe to retry, or what action to take without
reading PostHog/Langfuse.

Expose the remaining, already-computed classification fields:

- carry failureStage / failureRetryable / failureUserAction on the run
  (same source object as failureCategory/Detail);
- GET /api/runs/:id/diagnostics β€” a dedicated sub-resource so
  GET /api/runs/:id stays lean; returns RunDiagnosticsResponse with the
  full classification (`failure` is null for a run that did not fail);
- od run inspect <runId> [--json];
- contracts: RunDiagnosticsResponse + re-export RunFailureStage /
  RunFailureUserAction onto the API surface (mirrors the existing
  RunFailureCategory / RunFailureDetail re-exports).

Pure exposure: no new computation, no change to classification logic
(stays out of the accuracy-tuning lane, nexu-io#4966 / nexu-io#5223 / nexu-io#5356), and
complementary to nexu-io#5321 (which surfaces category/detail as chat guidance).
Availability matches GET /api/runs/:id (the live run registry). The
timing-segment and token/cache slices β€” which need a storage decision β€”
follow separately, and the read-only web diagnostics panel lands with
them once the shape is blessed.

Refs nexu-io#5489
@lefarcen
lefarcen requested a review from PerishCode July 13, 2026 02:34
@lefarcen lefarcen added size/M PR changes 100-300 lines risk/high High risk: apps/desktop, daemon, auth, migration, workflows, package deps type/feature New feature labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk/high High risk: apps/desktop, daemon, auth, migration, workflows, package deps size/M PR changes 100-300 lines type/feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants