feat(daemon): expose per-run failure diagnostics β od run inspect + endpoint (slice 1 of #5489) - #5511
Draft
tomsen02 wants to merge 1 commit into
Draft
feat(daemon): expose per-run failure diagnostics β od run inspect + endpoint (slice 1 of #5489)#5511tomsen02 wants to merge 1 commit into
tomsen02 wants to merge 1 commit into
Conversation
β¦dpoint) Slice 1 of nexu-io#5489. The daemon already computes the full failure classification at finalize-time for retry-policy + telemetry, but the API surface only carried failureCategory / failureDetail β failureStage, retryable, and user_action were telemetry-only, so a local user or an external agent driving Open Design through `od` could not see why a run failed, whether it is safe to retry, or what action to take without reading PostHog/Langfuse. Expose the remaining, already-computed classification fields: - carry failureStage / failureRetryable / failureUserAction on the run (same source object as failureCategory/Detail); - GET /api/runs/:id/diagnostics β a dedicated sub-resource so GET /api/runs/:id stays lean; returns RunDiagnosticsResponse with the full classification (`failure` is null for a run that did not fail); - od run inspect <runId> [--json]; - contracts: RunDiagnosticsResponse + re-export RunFailureStage / RunFailureUserAction onto the API surface (mirrors the existing RunFailureCategory / RunFailureDetail re-exports). Pure exposure: no new computation, no change to classification logic (stays out of the accuracy-tuning lane, nexu-io#4966 / nexu-io#5223 / nexu-io#5356), and complementary to nexu-io#5321 (which surfaces category/detail as chat guidance). Availability matches GET /api/runs/:id (the live run registry). The timing-segment and token/cache slices β which need a storage decision β follow separately, and the read-only web diagnostics panel lands with them once the shape is blessed. Refs nexu-io#5489
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #5489. Draft β a concrete reference implementation of the slice-1 shape @lefarcen and I converged on in the proposal, posted to make the product-direction review easier. Not asking to merge ahead of that direction.
Why
Per #5489: the daemon already computes the full failure classification at finalize-time (for retry-policy + telemetry), but the API surface only carried
failureCategory/failureDetail.failureStage,retryable, anduser_actionwere telemetry-only β so a self-hoster, or an external agent driving Open Design throughod, could not see why a run failed, whether it is safe to retry, or what action to take without reading PostHog/Langfuse. That breaks the UI+CLI dual-track rule and the CLI-as-embeddability-contract principle in AGENTS.md.What this slice does (backbone: contracts + daemon + CLI + test)
failureStage/failureRetryable/failureUserActionon the run (same source object as the existing category/detail,server.ts).GET /api/runs/:id/diagnosticsβ a dedicated sub-resource soGET /api/runs/:idstays lean; returnsRunDiagnosticsResponsewith the full classification (failureisnullfor a run that did not fail).od run inspect <runId> [--json].RunDiagnosticsResponse+ re-exportRunFailureStage/RunFailureUserAction(mirrors the existingRunFailureCategory/RunFailureDetailre-exports).The 2/5 β 5/5 split (per the proposal thread)
This carries the bounded failure classification only. Availability matches
GET /api/runs/:id(the live run registry), so it sidesteps the persistence seam entirely. The heavier timing-segment + token/cache breakdown β which is fire-and-forget into telemetry today and needs the storage decision (persist to SQLite under theRUNTIME_DATA_DIRcontract vs in-memory only) β is deliberately not in this slice.Deliberately deferred (call these out for review)
od run inspecthere + the panel in the follow-up.Scope boundaries
Pure exposure β no new computation, no change to classification logic (stays out of the accuracy-tuning lane #4966 / #5223 / #5356). Complementary to #5321 (which surfaces category/detail as chat guidance), not a duplicate.
Validation
pnpm --filter @open-design/daemon typecheck(both tsconfigs) β cleanpnpm guardβ 78/78apps/daemon/tests/run-diagnostics-endpoint.test.tsβ green 3/3: fails a run through a fake claude over the production HTTP API, asserts/diagnosticsnow carriesstage/retryable/userAction(control: the plain status endpoint does not).Happy to adjust the endpoint shape, field names, or the persistence/UI phasing once the product direction lands.
Surface area
od run inspectGET /api/runs/:id/diagnostics+RunDiagnosticsResponse