Skip to content

streaming: aborted cross-provider responses and ensemble streams report status 200 instead of 499 #1004

Description

@moonming

Two streaming paths report status_code: 200 on usage events for streams the client aborted mid-flight, where the equivalent single-model chat path correctly reports 499 (CLIENT_CLOSED_REQUEST). Both predate the trace-foundation work (verified present at 4a98a83); surfaced during an independent audit of the OTLP trace PR.

Path 1 — cross-provider /v1/responses streaming: the stream guard tracks reached_end, but the status choice only consults guardrail_blocked (422 vs 200), ignoring reached_end == false. An aborted stream therefore emits a terminal usage event with status_code: 200.

Path 2 — streaming ensemble: the judge-stream completion path hardcodes 200 regardless of whether the synthesized answer's stream reached EOF.

Impact: per-status telemetry (usage rows, http.response.status_code on exported spans, status-labelled Prometheus counters) counts aborted streams as successes on these two paths, inconsistent with the single-model chat path.

Expected precedence, matching the chat stream guard: !reached_end → 499; guardrail block → 422; otherwise 200. Worth a paced-stream abort e2e per path when fixing — the existing suites only drain streams to EOF.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Long-tail integrations — backlogbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions