Preserve canonical metadata, citations, and stream responses#180
Merged
Conversation
Give transport boundaries canonical metadata codecs and let Bedrock replay provider-authored citation blocks without flattening or losing source locations.
Remove the open-ended metadata side channel so clean EOF exposes canonical content, usage, and stop reason through one typed response contract.
Honor ToolChoiceNone only when no tools are advertised so callers never mistake Bedrock auto-selection for a no-tools guarantee.
Describe the invariant in terms of forbidden budgeted work so provider history catalogs and terminal bookkeeping calls remain explicit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Streamer.Response()the sole terminal stream state and remove the open-ended metadata side channelToolChoiceNonewhen tools are defined instead of silently treating suppression as auto-selectionWhy
A completed model turn must survive transport, tracing, and replay without reconstruction. Generic metadata transport can coerce numbers, cited Bedrock answers previously failed on the next turn, and a second stream metadata channel allowed terminal usage and citations to diverge from the canonical response. Bedrock also has no provider-level “none” mode when a tool catalog is present, so accepting that request created a false guarantee and allowed forbidden terminal calls.
How
model.MarshalMetadataandmodel.UnmarshalMetadatapreserve JSON numbers and nested provider data exactly. Bedrock rebuilds validated native citation blocks and rejects unsupported forms explicitly. Streaming providers publish content, usage, and stop reason once throughResponse()after clean EOF;UsageChunkremains the progressive accounting event while a stream is active. Tool suppression now succeeds only for requests with no advertised tools and otherwise fails fast with a precise capability error.Test plan
make testmake lintgit diff --check