Skip to content

Preserve canonical metadata, citations, and stream responses#180

Merged
raphael merged 4 commits into
mainfrom
fix/canonical-metadata-citation-replay
Jul 13, 2026
Merged

Preserve canonical metadata, citations, and stream responses#180
raphael merged 4 commits into
mainfrom
fix/canonical-metadata-citation-replay

Conversation

@raphael

@raphael raphael commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

  • preserve provider-authored message metadata with lossless canonical JSON codecs
  • replay Bedrock citations natively across character, chunk, and page locations
  • make Streamer.Response() the sole terminal stream state and remove the open-ended metadata side channel
  • reject Bedrock ToolChoiceNone when tools are defined instead of silently treating suppression as auto-selection

Why

A completed model turn must survive transport, tracing, and replay without reconstruction. Generic metadata transport can coerce numbers, cited Bedrock answers previously failed on the next turn, and a second stream metadata channel allowed terminal usage and citations to diverge from the canonical response. Bedrock also has no provider-level “none” mode when a tool catalog is present, so accepting that request created a false guarantee and allowed forbidden terminal calls.

How

model.MarshalMetadata and model.UnmarshalMetadata preserve JSON numbers and nested provider data exactly. Bedrock rebuilds validated native citation blocks and rejects unsupported forms explicitly. Streaming providers publish content, usage, and stop reason once through Response() after clean EOF; UsageChunk remains the progressive accounting event while a stream is active. Tool suppression now succeeds only for requests with no advertised tools and otherwise fails fast with a precise capability error.

Test plan

  • focused model, provider, runtime, and gateway tests
  • make test
  • make lint
  • git diff --check

raphael added 2 commits July 13, 2026 01:20
Give transport boundaries canonical metadata codecs and let Bedrock replay provider-authored citation blocks without flattening or losing source locations.
Remove the open-ended metadata side channel so clean EOF exposes canonical content, usage, and stop reason through one typed response contract.
@raphael raphael changed the title Preserve metadata and cited response replay Preserve canonical metadata, citations, and stream responses Jul 13, 2026
raphael added 2 commits July 13, 2026 02:42
Honor ToolChoiceNone only when no tools are advertised so callers never mistake Bedrock auto-selection for a no-tools guarantee.
Describe the invariant in terms of forbidden budgeted work so provider history catalogs and terminal bookkeeping calls remain explicit.
@raphael
raphael merged commit 89b93da into main Jul 13, 2026
2 checks passed
@raphael
raphael deleted the fix/canonical-metadata-citation-replay branch July 13, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant