Skip to content

Normalize completion responses at the provider boundary and erase the model type at agent construction - #2257

Merged
gold-silver-copper merged 20 commits into
mainfrom
refactor/normalized-model-swapping
Aug 5, 2026
Merged

Normalize completion responses at the provider boundary and erase the model type at agent construction#2257
gold-silver-copper merged 20 commits into
mainfrom
refactor/normalized-model-swapping

Conversation

@gold-silver-copper

@gold-silver-copper gold-silver-copper commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Normalizes completion responses at the provider boundary and erases the agent's model type once at construction, enabling runtime model swapping.

Supersedes three PRs:

Phase 1 — normalization (from #2256, with fixes)

  • CompletionResponse / StreamFinal are concrete, carrying usage, finish_reason, provider, model, and two distinct IDs: message_id (only IDs a provider would recognize on a replayed assistant message — Anthropic/Responses msg_*; what agent history promotes) and response_id (response-scoped — chatcmpl-*, Gemini responseId, Cohere generation IDs; telemetry/logging only, never replayed). This stops chat response IDs being echoed to the Responses API as fabricated output-message IDs — critical once models can swap between turns.
  • Terminal-emission contract (documented on StreamFinal): a terminal record is emitted only when the provider's genuine end-of-response event arrived and no transport error occurred. Truncated streams yield their content (including fully-delivered tool calls) and simply end. SSE parse errors are surfaced without aborting the stream, so a later genuine terminal still completes it. Regression-tested for Anthropic, Gemini REST, the shared OpenAI chat path, and Cohere (EOF-truncation / parse-error-then-EOF / parse-error-then-terminal / transport-error).
  • ConstructCompletionModel is public and documented as the extension point, with a compile probe showing an out-of-tree provider extension reaching the blanket CompletionClient impl over Client<Ext, H> (orphan rules prevent any other route).
  • Finish-reason mappers preserve unknown wire values verbatim as Other(String); the Gemini REST and gRPC mappers now agree on FINISH_REASON_UNSPECIFIED (absent); the shared streaming entry takes a runtime provider name; the failure-status policy is documented once on FinishReason.
  • Typed provider-native access is preserved via inherent raw_completion / raw_stream on every provider model — one request either way.

Phase 2 — agent erasure and routing (from #2252, with fixes)

  • AgentBuilder::new erases any CompletionModel + 'static into a concrete ModelHandle (itself a CompletionModel). Agent, AgentRunner, prompt/stream requests, and Extractor lost their model parameter. Erasure is lossless — no serde_json::Value finals; model.rs went from 358 lines to 175. ProviderCapabilities is captured by value at erasure (the stored Arc<dyn Fn() -> bool> is gone).
  • ModelSelection hook event on the existing AgentHook stack (per refactor(agent): support hook-driven runtime model routing #2252's design), with the ordering fixed: CompletionCall hooks → merged RequestPatch (now a documented request_patch field on ModelSelection) → selection hooks → prepare against the selected handle's capabilities → advance previous_model immediately before invoking the model. Stopped or failed preflight never advances previous_model; an issued attempt counts even if the provider errors. Parity-tested on both run and stream.
  • Runtime swapping surfaces: Agent::set_model, per-run using_model(...), on_model_select, plus a routing example and a 20-test integration suite.

Verification

  • cargo test --workspace --all-features: 156 suites, 0 failures (rig-core lib 1014, rig-agent 464 lib + 20 swapping + 12 doc + 3, all provider cassette suites green).
  • cargo clippy --workspace --all-targets --all-features, cargo fmt --check, scoped cargo doc -D warnings, cargo check --target wasm32-unknown-unknown -p rig-core -p rig-agent -p rig: clean.
  • Cassette request invariant: git diff 6cfae6d8..HEAD -- tests/cassettes is empty — no request matcher or fixture changed.
  • Independently reviewed against the settled design contracts (ID split, terminal emission, public construction seam, mapper unification, erasure/ordering); all findings addressed in ec9f2625.

MIGRATING.md documents every break: the concrete response types and ID contract, raw_completion/raw_stream, GetTokenUsage removal and the record_token_usage(&Usage) signature, the terminal-emission behavior change, required CompletionClient::completion_model, capabilities(), and the agent-side model-parameter removal.


Update — heads 5b521ef0 and 5b939580

Later revisions supersede parts of the description above:

  • 5b521ef0 replaced the closure-pair ModelHandle internals with a private object-safe ErasedModel trait (single Arc allocation via unsize coercion) and removed the Clone supertrait from CompletionModel (breaking; completion_request gates on Self: Clone; send/stream no longer clone the model). Earlier, three ollama cassettes were deliberately rerecorded because streamed tool-call ids changed from "" to the function name — the "cassette diff is empty" claim above no longer holds.
  • 5b939580 addresses the full third-round review: WS/copilot response.incomplete terminals preserve partial output; bedrock emits parallel tool calls (keyed by content_block_index); corrupt stream frames surface as Err items while unrecognized valid-JSON events are skipped for forward compatibility; errored streams flush fully-delivered tool calls; gemini REST/Interactions enums preserve unknown wire values verbatim; CompletionResponse::finish_reason is private behind a getter; CompletionModel is implemented for Arc<M>; empty-string ids/models are treated as absent.

Verification at 5b939580: cargo test --workspace --all-features green (all cassette suites), cargo clippy --workspace --all-targets --all-features -- -D warnings clean, cargo fmt --check clean, wasm cargo check for rig-core/rig-agent/rig clean.

5c73639c — reverified fourth-round review addressed

  • A full reasoning block supersedes its accumulated deltas in the aggregated streaming choice (previously duplicated).
  • The plain-OpenAI chat profile surfaces unparseable frames as Err items — the saw_any_valid_frame gate can no longer yield a completely silent empty stream.
  • Bedrock's MessageStop straggler flush is gated on a ToolUse stop reason (no fabricated {}-args calls or spurious errors on max-tokens truncation).
  • One error-flush ordering everywhere (ToolCall… → Err → end, no terminal): shared compat, Responses SSE, and copilot previously had three different behaviors; documented on the StreamFinal contract table.
  • Stream parse policy now discriminates on the known event type (known + schema defect → Err; unknown type → skipped for forward compatibility); the ChatGPT buffered fallback fails on corrupt known frames; xai's reasoning_summary_text.done (text, not delta) decodes.
  • CompletionResponse/StreamFinal deserialize through wire-shape mirrors funneling the validating setters (reconciliation + empty-string filtering hold for persisted values; wire format unchanged).
  • WS/SSE replay merges terminal-body message text absent from deltas; gemini InteractionStatus::is_terminal uses a known-in-flight-states allowlist so unknown statuses read as terminal.

Verification at 5c73639c: cargo test --workspace --all-features green (all cassette suites, no cassette edits), clippy --all-targets --all-features -D warnings clean, fmt --check clean, wasm checks clean.

34ee8ba5 — reverified fifth-round review addressed

  • Delta-less streamed choices (Azure's prompt_filter_results prelude) parse as no-op frames — no more spurious error item on every Azure stream (openai-compatible + copilot chat models).
  • Unmodeled Responses content_part shapes (refusal/reasoning_text parts) parse as no-ops — refusal turns no longer error on SSE or hard-fail the ChatGPT buffered route; refusal text flows via response.refusal.delta.
  • Reasoning supersede is genuinely id-correlated: strict identity table plus a by-id fallback scan, so interleaved output (reasoning → tool call → completed block) replaces instead of duplicating; contract documented on StreamedAssistantContent::Reasoning and in MIGRATING.
  • Whitespace-only streamed text no longer duplicates the terminal body's message; gemini interactions RequiresAction is terminal-for-the-poll and documented as a distinct resumable outcome.

Verification: full workspace test suite with all features green (all cassette suites, no cassette edits), clippy --all-targets --all-features -D warnings clean, fmt clean, wasm checks clean.

… boundary

Remove the response and construction associated types from `CompletionModel`
so the trait describes only what a model *does*, and make the ordinary
completion and streaming types concrete.

`CompletionResponse` loses `raw_response` and gains the metadata callers
actually reached into it for: a normalized `finish_reason`, the stable
`provider` descriptor name, the provider-reported `model`, and `message_id`.
Provider-native payloads stay reachable through each model's inherent
`raw_completion`/`raw_stream`, which share one request, transport, parser,
telemetry path, and error path with their normalized counterparts.

Three details are load-bearing:

- The `Stop` -> `ToolCalls` reconciliation lives in exactly one place
  (`FinishReason::reconcile_with_output`) and is applied by both the unary
  setters and `normalize_stream`. Several OpenAI-compatible gateways report a
  bare `stop` on a tool-calling turn, so a caller branching on `ToolCalls`
  would otherwise miss the call on the unary path while the streaming path
  caught it.
- Conversions over a wire type shared by many providers take the descriptor
  name as an input rather than hardcoding one, so a shared shape cannot
  mislabel the provider that used it.
- `StreamingCompletionResponse` records its provider when the stream is
  opened, so a stream that errors or truncates before its terminal record
  still reports one.

Construction moves to the required `CompletionClient::completion_model`;
`ProviderCapabilities` replaces `composes_native_output_with_tools` with plain
data a runtime can snapshot. Request construction is unchanged — no cassette
fixture differs.
…dary

Assertions that read normalized metadata now read the fields that replaced
`raw_response` (`model`, `finish_reason`, `provider`, `message_id`).
Raw-vs-normalized parity tests keep their intent by taking the provider's own
response from the model's inherent `raw_completion` and normalizing that same
value, rather than issuing a second request — each cassette holds exactly one
interaction and fails on both unmatched extra requests and unconsumed ones.

Tests that read genuinely provider-specific stream payloads (reasoning
metadata, local generation counters) move to `raw_stream`, which is what that
escape hatch is for.

No cassette fixture changed.
/responses answers with an SSE body even for a non-streaming request, so the
value raw_completion returns is reassembled from the terminal event — and that
event sometimes carries an empty output, with the content living only in the
preceding events. Point callers who need full provider fidelity at raw_stream
rather than letting them discover this from an empty field.
…dary

An adversarial review of the full diff raised 38 findings; 24 survived
refutation. The load-bearing ones:

Normalization was stated as `TryFrom<(&str, Response)>`, which the orphan rule
makes unimplementable outside rig-core — a tuple is not a local type, so no
other crate could satisfy the bound. That silently blocked external provider
extensions, which are explicitly in scope. It is now a
`NormalizeCompletionResponse` trait, implementable on any provider's own
response type anywhere.

`normalize_stream` counted tool-call *deltas* toward the Stop -> ToolCalls
upgrade while the unary path counts only completed calls, so a stream whose
tool call never assembled reported `ToolCalls` over a choice containing none —
the exact divergence the single-implementation rule is supposed to prevent.
Both paths now count only completed calls.

`send_compatible_streaming_request` is public and stamped a hardcoded "openai"
onto every stream it produced, reintroducing in a public helper the same
placeholder hazard this change removes elsewhere. It takes the descriptor name
now.

Vertex AI built `FinishReason::Other` by `Debug`-formatting the SDK enum, which
turned MALFORMED_FUNCTION_CALL into MALFORMEDFUNCTIONCALL. It uses the SDK's
own `name()`.

Also reverted two behavior changes that had crept in and do not belong to a
normalization change: `message_id` was being fed response-scoped ids
(`chatcmpl-`, Gemini `responseId`) on five paths even though the agent replays
that field into assistant history, and Cohere's usage silently moved from
`billed_units` to `tokens`. Both are back to the pre-existing semantics.

Three confirmed findings are deliberately untouched because they are identical
at the merge base: anthropic's SSE parse-failure guard, bedrock's MaxTokens
early yield, and Z.AI's two descriptor names.
Both were cases of acting on a finding's mechanism without first checking
whether it described a reachable harm. The test suite caught both.

`message_id` was stripped from five providers on the claim that a
response-scoped id would corrupt agent history, since the agent writes it into
`Message::Assistant { id }`. That data path is real, but every provider's
request conversion drops the id before it reaches the wire (see
providers/openai/completion/mod.rs, `id: None`), so nothing is replayed.
Stripping it discarded identifiers this change is meant to preserve. They are
restored, and the field now documents what it actually holds — providers differ
on whether it names the message or the response — and that it is not echoed
back.

Cohere's usage source had been changed to prefer `tokens`; reverting to
`billed_units` kept every existing number stable but left rig reporting zero
usage — the documented "no metrics" sentinel — for a response carrying `tokens`
and no `billed_units`. Neither was right. `billed_units` stays primary so no
caller's numbers move, with `tokens` as a fallback only where rig previously
reported nothing.

Gemini's Interactions API keeps its continuation handle reachable through
`raw_completion`, which is where the pre-refactor tests read it from
(`raw_response.id`); it is a `previous_interaction_id` handle, not an assistant
message, so the escape hatch is its proper home.
…message IDs

CompletionResponse and StreamFinal gain a response_id field for identifiers
that name the response as a whole (OpenAI chat chatcmpl-*, Gemini responseId,
Cohere generation IDs, Gemini Interactions interaction IDs). message_id is now
reserved for identifiers the provider would recognize on a replayed assistant
message (Anthropic msg_*, OpenAI Responses output-message msg_*), which is what
agent history promotes into Message::Assistant's id. This stops chat response
IDs from being replayed to the Responses API as fabricated output-message IDs.
ConstructCompletionModel was pub(crate), but the blanket CompletionClient impl
over Client<Ext, H> bounds on it, and the orphan rule prevents a downstream
crate from implementing CompletionClient for the foreign Client<Ext, H> type.
Together that locked out-of-tree generic provider extensions out of
completion_model entirely. The hook is now public and documented as the
extension point, with a compile probe showing an external extension reaching
the blanket impl through public API only.
A synthetic FinalResponse was still emitted when a stream reached EOF without
the provider's genuine end-of-response event, and (on some paths) after a
yielded SSE parse error. Every provider stream now tracks whether the real
terminal signal arrived — Anthropic message_delta with stop_reason, OpenAI
[DONE]/finish_reason chunk, Responses response.completed, Cohere message-end,
Gemini finishReason chunk (REST, gRPC, Interactions), Copilot
response.completed — and emits the terminal record only then. Fully delivered
tool calls still flush on truncation; parse errors are surfaced without
aborting the stream, so a later genuine terminal can still complete it.
Regression tests cover EOF-truncation, parse-error-then-EOF, and
parse-error-then-terminal for Anthropic, Gemini REST, the shared OpenAI chat
path, and Cohere. The contract is documented on StreamFinal.
…boundary contract

- send_compatible_streaming_request takes the provider name as impl
  Into<String>, so runtime-named gateways can use the shared path.
- The Gemini REST finish-reason mapper now treats FINISH_REASON_UNSPECIFIED
  as absent (None), matching the gRPC mapper's handling of the same wire
  value instead of surfacing an Other("FINISH_REASON_UNSPECIFIED") that the
  sibling transport would report as no reason.
- FinishReason documents the single policy for provider failure statuses that
  arrive with parseable output.
Erase the typed CompletionModel once, at AgentBuilder::new/ExtractorBuilder::new,
into an opaque, cloneable ModelHandle that is itself a CompletionModel with its
ProviderCapabilities captured by value at erasure. Agent, AgentBuilder (after
new()), AgentRunner, prompt/stream requests, and Extractor carry no model type
parameter. Port the ModelSelection hook event and ModelSelectionAction onto the
AgentHook stack (registration-order chaining, last selection wins, stop is
terminal), per-run using_model defaults, run-local extractor overrides, the
runtime_model_routing example, and the runtime_model_swapping suite.
Reorder the model-call boundary: completion-call hooks resolve first; only if
they proceed does ModelSelection fire, carrying the merged per-turn patch on a
new public request_patch field; request preparation then runs against the
selected model's captured ProviderCapabilities before the attempt is issued.
previous_model now advances immediately before invoking the selected model's
unary or streaming operation, so an issued attempt counts even when the
provider errors, while a completion-call stop, selection stop, or preparation
failure leaves it untouched. Document the ordering contract on both hook
events and add blocking/streaming parity tests covering the merged patch,
patch-driven routing, stop/preparation-failure suppression, and issued-attempt
tracking.
- The shared OpenAI chat-compatible stream no longer aborts on a chunk parse
  error: the error is surfaced and consumption continues, matching the
  Anthropic/Gemini parse-error policy, with the missing terminal signal still
  suppressing the terminal record on truncation.
- The Responses API streaming terminal now carries the resp_* response ID
  (accumulator, Copilot inline path), matching the unary path.
- MIGRATING covers the agent-side model-type-parameter removal and runtime
  model swapping, and no longer claims an Agent's model is fixed.
Correctness:
- Reject provider-stream EOF without a terminal record as truncation in
  the streaming agent, before usage fallback, history commit, or tool
  dispatch; add text-only and tool-call truncation tests.
- Carry response_id on OpenRouter unary, the ChatGPT empty-output
  fallback, gemini-grpc unary, and vertexai unary responses.
- Add #[serde(default)] to DeepSeek Usage and to all Option fields of
  CompletionResponse/StreamFinal.
- Route the mock model's stream through normalize_stream so tests see
  the same Stop -> ToolCalls reconciliation as real providers.

Pre-existing contract violations:
- ollama: skip malformed NDJSON lines instead of aborting; use the
  function name as the streamed tool-call id.
- bedrock: MAX_TOKENS ends as a genuine Length terminal; parse failures
  warn and skip; Metadata always emits the terminal record.
- copilot: use Usage::to_normalized (saturating, real completion and
  reasoning tokens) on the unary path and in telemetry.
- vertexai: map cached_content_token_count instead of hardcoding zero.

Coverage and tooling:
- Nine new streaming tests: core mid-stream-error/truncation, gemini
  REST transport error, cohere malformed frame, OpenAI Responses
  truncated EOF, anthropic transport error, xai/gemini-interactions/
  ollama truncation.
- rig-agent dev-dependencies enable rig-core test-utils so plain
  `cargo test -p rig-agent` builds.
- Candle real-model test exercises the normalized completion()/stream()
  surfaces.
- Raise malformed-SSE-frame logs to warn.

Docs: fix the stale gemini example narrative, add StreamFinal
response_id and the normalize-bridge pointer to MIGRATING.md, remove
dangling/duplicated doc blocks, re-section changelogs, add
ModelSelection::new for hook unit tests, document ModelHandle's
per-call clone, and make the previous_model comment precise.
CI blockers:
- Rerecord three ollama cassettes (local qwen3:4b) whose requests still
  expected the old empty streamed tool-call ids; tool results now carry
  the function name as tool_name, matching the unary path.
- Convert bedrock's one-arm MessageStop match to an if, fixing
  clippy::single_match under --all-features -D warnings; reflow the
  kill-token example's doc list for doc_overindented_list_items.

Parity:
- Treat OpenAI Responses `response.incomplete` as a genuine terminal on
  the SSE and WebSocket paths (partial output, usage, and a
  map_finish_reason-mapped finish reason), matching the unary path;
  `response.failed` remains an error. Pinned tests rewritten and a
  WebSocket incomplete-terminal test added.

Hardening:
- ModelHandle retains the erased model in one shared Arc and clones only
  the Arc per attempt, so interior-mutable model state persists across
  attempts; handle docs state the retention guarantee.
- MIGRATING documents the agent-level truncation error for streams that
  end without a terminal record.
- Remove a stray blank line in StreamingCompletionResponse.
…e supertrait

Erasure (rig-agent):
- Replace ModelHandle's boxed closure pair with a private object-safe
  ErasedModel trait and a blanket impl over CompletionModel + 'static —
  the tower::BoxService shape: one vtable, borrowed futures delegating
  to the RPITIT methods, no per-attempt model handling beyond Box::pin.
- The WasmCompat supertraits carry the wasm cfg fork (as ErasedTool
  does), removing all four cfg-forked callback type aliases.
- Store capabilities snapshot, label, and the unsized erased model in a
  single Arc<ModelDriver<dyn ErasedModel>> allocation.
- Pin the shared-instance invariant structurally: a clone-counting probe
  model asserts zero model clones across erasure and repeated
  completion/stream attempts.

Trait relaxation (rig-core, breaking for generic code):
- CompletionModel no longer requires Clone; the trait demands only
  async service behavior, tower::Service-style. completion_request
  gates on `where Self: Sized + Clone`.
- build/send/stream funnel through one private into_model_and_request
  destructuring, removing the terminal methods' model clones while
  keeping build()'s public behavior identical.
- tool_choice_modes in the conformance suite bounds + Clone explicitly,
  matching its siblings; every other call site is a concrete Clone type
  and compiles unchanged.
- MIGRATING and the rig-core changelog document the relaxation.

Verified: workspace check --all-features, wasm32-unknown-unknown check
for rig-core/rig-agent/rig, fmt, clippy --all-features -D warnings,
rig-agent suite (incl. runtime_model_swapping and conformance), and
rig-core --features test-utils all green. No cassettes touched.
Correctness (P1):
- openai responses websocket: streamed item events now feed the same
  RawChoiceAccumulator the SSE path uses, so a response.incomplete
  terminal with an empty output array keeps the partial output that
  streamed before it, instead of failing with "Response contained no
  message or tool call".
- copilot responses streaming: response.incomplete is a genuine
  terminal (partial content, Length finish reason), no longer a vague
  stream error with no terminal record; only response.failed errors.
- bedrock streaming: in-flight tool calls are keyed by
  content_block_index and flushed at ContentBlockStop, so parallel tool
  calls all arrive (previously a single slot kept only the last), text
  after a closed tool block is delivered, and malformed tool-call JSON
  surfaces an Err item instead of silently emitting a ToolCalls
  terminal with zero calls.

Streaming contract (P2):
- corrupt frames (invalid JSON) yield an Err item and the stream keeps
  consuming — openai responses, copilot chat, cohere, ollama; a later
  genuine terminal still completes the stream. Valid-JSON events with
  unrecognized shapes are skipped for forward compatibility with new
  provider event types (xAI's extra Responses events exercised this).
- errored streams flush fully-delivered tool calls before ending
  (shared openai-compatible path and the copilot responses route),
  matching the clean-EOF truncation path.
- a bare [DONE] after only unparseable frames no longer fabricates a
  zero-usage terminal record.
- gemini REST/Interactions wire enums gained untagged Unknown(String)
  catch-alls so unknown values are preserved verbatim as
  FinishReason::Other, matching the gRPC mapper, instead of failing the
  whole payload.
- cohere message-end without a delta still emits the terminal record.
- the StreamFinal emission contract now documents the three failure
  shapes (transport error / recoverable parse error / truncation) and
  that consumers must drain to None.

API (breaking, documented in MIGRATING):
- CompletionResponse::finish_reason is a private field with a getter,
  so the Stop -> ToolCalls reconciliation cannot be bypassed by direct
  assignment.
- CompletionModel is implemented for Arc<M> by forwarding, making the
  "wrap it in an Arc" guidance real through the generic APIs.
- identifier/model setters on CompletionResponse and StreamFinal treat
  empty strings as absent, centralizing a rule previously duplicated at
  provider call sites.

Tests: parallel-tool-call and text-after-tool bedrock coverage, WS
incomplete-with-empty-output regression, copilot incomplete terminal,
malformed-frame surfacing per provider, unknown-wire-value round trips,
a NonCloneModel compile-time traits() probe, and the clone-probe
assertions split per path. The misleading "did not yield
response.completed" error now names both terminal events, and the stale
Clone doc line on CompletionClient is rewritten.
Correctness:
- streaming aggregation: a full reasoning block supersedes its
  accumulated deltas (correlated by reasoning item id; distinct ids
  still append), so the aggregated choice and agent history no longer
  carry the same reasoning twice.
- openai chat default profile: unparseable frames surface as Err items
  instead of Ok(None), so the saw_any_valid_frame gate can no longer
  produce a completely silent empty stream on the plain OpenAI path.
- bedrock: the MessageStop straggler flush is gated on a ToolUse stop
  reason — a tool block truncated by MaxTokens is dropped with a
  warning rather than fabricated into a {}-args call or a spurious
  error item.
- openai responses websocket/SSE replay: terminal-body message text is
  merged per content kind when it never streamed as deltas (the
  reasoning-deltas-plus-body-text sequence previously dropped the
  answer); the untested incomplete-with-body-output quadrant is now
  pinned.

Streaming contract:
- one flush ordering on every path (shared compat, openai responses
  SSE, copilot responses): fully-delivered tool calls are yielded
  before the terminal Err, nothing follows the error, no terminal
  record — previously the three paths had three behaviors and the
  flush was invisible to first-Err consumers. Documented on the
  StreamFinal contract table.
- parse policy discriminates on the known event type at every site
  (responses, copilot chat, cohere, openai chat default): a known
  event with a schema defect surfaces an Err; unknown event types are
  skipped for forward compatibility. The ChatGPT buffered SSE fallback
  fails the completion on corrupt known frames instead of returning
  silently partial content. xai's reasoning_summary_text.done (which
  carries text, not delta) now decodes via a serde alias.
- CompletionResponse and StreamFinal deserialize through wire-shape
  mirrors that funnel the validating setters, so finish-reason
  reconciliation and empty-string filtering also hold for persisted
  values; serialization is unchanged and round-trips are identity.
- gemini interactions InteractionStatus::is_terminal enumerates the
  known in-flight states (semantic-kernel-style allowlist), so an
  unknown status reads as terminal.

Cleanups: redundant per-provider empty-string filters removed
(deepseek, openai, copilot); ollama post-terminal behavior pinned by
test.
Two regressions from the previous round's known-type parse policy,
both the same root cause — the discriminator lists were broader than
the typed wire models:

- Delta-less streamed choices (Azure prepends a prompt_filter_results
  chunk to every stream when content filtering is enabled, its
  default) parsed as recognizable-but-broken and surfaced a spurious
  error item on every Azure request. The choice's delta is now
  serde-defaulted (openai-compatible and copilot chat chunk models),
  so the prelude parses as the no-op frame the reference SDKs treat it
  as (vercel nullish + skip, langchain return-None, semantic-kernel
  documented skip).
- Unmodeled Responses content_part shapes — the refusal and
  reasoning_text parts real refusal/reasoning-text turns emit — failed
  the typed decode: spurious error items on SSE, and a hard failure of
  the whole ChatGPT buffered completion (the part frame precedes the
  deliverable refusal deltas). ContentPartChunkPart gained an untagged
  Unknown(Value) catch-all (the Output::Unknown shape); the content
  still flows via response.refusal.delta.

Reasoning aggregation: the full-block supersede is now genuinely
id-correlated. The identity table is strict (matching ids or both
absent replace; an id on only one side appends instead of clobbering
an unrelated item), and when other output cleared the active index
(reasoning -> tool call -> completed block) a by-id fallback scan
replaces instead of duplicating. The supersede contract is documented
on StreamedAssistantContent::Reasoning and in MIGRATING: a full block
replaces accumulated deltas with the same id.

Also: the websocket/SSE terminal-body merge gates on any streamed text
(is_empty, not trim) so all-whitespace turns don't duplicate; gemini
interactions RequiresAction is terminal for a poll loop — it never
advances without submitted tool results — and documented as a
distinct resumable outcome (semantic-kernel's taxonomy).

Tests: Azure filter-prelude no-op, refusal content-part frames on the
live and buffered paths, interleaved reasoning supersede, id-less
block append, RequiresAction terminality.
@gold-silver-copper
gold-silver-copper added this pull request to the merge queue Aug 5, 2026
Merged via the queue into main with commit 4f0bc70 Aug 5, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant