Skip to content

Classify empty provider streams as retryable#183

Merged
raphael merged 1 commit into
mainfrom
fix/bedrock-empty-stream
Jul 15, 2026
Merged

Classify empty provider streams as retryable#183
raphael merged 1 commit into
mainfrom
fix/bedrock-empty-stream

Conversation

@raphael

@raphael raphael commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Introduce model.ErrEmptyStream + model.NewEmptyStreamError, pairing a detection sentinel with a retryable unavailable ProviderError (code empty_stream).
  • Bedrock and Anthropic stream adapters classify a message stop without an active message — and event streams that close before any message starts — as empty streams instead of opaque protocol errors. Streams that close mid-message become retryable truncated_stream provider errors; duplicate message stops remain hard protocol errors.
  • Document the provider stream integrity contract in DESIGN.md and the new sentinel in docs/runtime.md.

Rationale

Bedrock intermittently terminates ConverseStream responses without ever emitting messageStart when the model produces an empty completion (observed in production on Haiku 4.5: messageStop first, zero output tokens, reproducible per transcript). The strict state machine introduced in v0.62 correctly rejects the shape, but callers could not distinguish it from a corrupted stream, so retry middleware had nothing to key on. Adapters still never fabricate a response; retry policy stays with the integrating application.

Test evidence

  • make lint: 0 issues.
  • make test: full suite green.
  • New coverage: processor-level empty-stream/duplicate-stop classification (Bedrock), eventless and stop-without-start streams (Anthropic), streamEndedEarlyError table test, NewEmptyStreamError contract test.

Bedrock intermittently terminates ConverseStream responses without ever
starting an assistant message when a model emits an empty completion
(observed in production on Haiku 4.5 as messageStop with no prior
messageStart and zero output tokens). The strict stream state machine
correctly rejects the shape but surfaced it as an opaque protocol error
that retry middleware could not detect.

Introduce model.ErrEmptyStream plus model.NewEmptyStreamError, which pair
the sentinel with a retryable unavailable ProviderError (code
empty_stream). The Bedrock and Anthropic adapters classify message stop
without an active message and event streams that close before message
start through it, and classify streams that close mid-message as
retryable truncated_stream provider errors. Duplicate message stops stay
hard protocol errors. Adapters still never fabricate responses; retry
policy remains with the integrating application.
@raphael
raphael merged commit 3fa8597 into main Jul 15, 2026
2 checks passed
@raphael
raphael deleted the fix/bedrock-empty-stream branch July 15, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant