Canonical model transcripts: exact token counting, tool naming, health, and session lifecycle#185
Merged
Merged
Conversation
Give transport boundaries canonical metadata codecs and let Bedrock replay provider-authored citation blocks without flattening or losing source locations.
Remove the open-ended metadata side channel so clean EOF exposes canonical content, usage, and stop reason through one typed response contract.
Honor ToolChoiceNone only when no tools are advertised so callers never mistake Bedrock auto-selection for a no-tools guarantee.
Describe the invariant in terms of forbidden budgeted work so provider history catalogs and terminal bookkeeping calls remain explicit.
Decode Bedrock's canonical over-context CountTokens response as the exact measurement so history compression can run without heuristic estimates.
Split the Anthropic adapter into a canonical request encoding shared by
Complete, Stream, and CountTokens plus a completion-policy layer, so token
counting no longer inherits completion-only requirements (max_tokens) and
cannot drift from the completion encoding (empty tools serialization).
Attach the tool-examples beta header additively (WithHeaderAdd) so
caller-configured betas survive, and only when authored examples are
present. Live-verified against Vertex rawPredict: Claude-on-Vertex delivers
input_examples natively with no beta activation and ignores the header, so
no transport gate is needed.
Own the token-counting projection once as model.CountingRequest; the
anthropic, bedrock, and vertex counters all consume it, deleting vertex's
divergent private copy. Make features/model/toolname the canonical
provider-name projection for the shared [a-zA-Z0-9_-]{1,64} contract:
openai delegates to it and toolname.BuildMaps owns the injectivity
invariant for anthropic, bedrock, and openai.
…tion lane Extract computeToolsetHealth as the pure owner of the staleness rule with explicit winner selection, so ProviderID no longer depends on replicated-map iteration order, and pin the staleness-threshold derivation with deriveStalenessThreshold so a config change cannot silently shrink the window providers have to respond. Fast table tests cover the rule; the Redis-backed tests move behind the integration build tag. The integration tag is an explicit opt-in to Docker-backed tests, so TestMain now fails loudly when Redis cannot be provisioned instead of skipping every test and turning the lane vacuously green. CI invokes make itest so the integration lane has one source of truth.
Make the durable session status the authority that stops in-flight runs: every planner activity (start and resume) consults it first and refuses to plan under an ended session, recording CancellationReasonSessionEnded so the terminal RunCompleted event carries canonical provenance even when engine cancellation never delivered. The workflow maps the refusal through the existing cancellation terminal path, demoting engine cancellation to an expedite-only optimization. Remove the timeout retry hint: elapsed execution time is not an instruction to repeat a call, so timeout and exhausted-budget failures are terminal for the current run. Collapse the policy retry vocabulary onto the planner enum via type aliases, deleting the mirrored constants and the runtime's deep-cloning conversion. Preserve stream event identity through the hinting sink, and cancel pending agent-tool child workflows at the parent finish-by deadline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Canonical model transcripts end to end: exact Anthropic token counting, one provider tool-name projection, deterministic registry health, and durable session-lifecycle enforcement in the run loop.
Model providers
CountTokensconsumes the encoding directly, so counting works without a defaultMaxTokensand can never drift from the completion encoding ("tools": []vs omitted).WithHeaderAdd) so caller-configured betas (e.g. 1M context) survive, and only when a tool authors examples — header-compatible gateways reject unknown beta identifiers. Live-verified against Vertex rawPredict (usage.input_tokensdeltas): Claude-on-Vertex deliversinput_examplesnatively with no beta activation and ignores the header, so no transport gate exists.model.CountingRequestis the single owner of the TokenCounter projection (thinking stripped, thinking-only messages dropped); anthropic, bedrock, and vertex all consume it, deleting vertex's divergent private copy.features/model/toolnameowns the provider-name projection for the shared[a-zA-Z0-9_-]{1,64}contract (Claude Messages API + OpenAI Responses); openai's byte-identical copy is deleted andtoolname.BuildMapsowns the injectivity invariant for anthropic, bedrock, and openai. Direct Anthropic now sends fully namespaced tool names (leaf-stripping removed); provider names are per-request projections, never persisted.Registry
computeToolsetHealthis a pure function with explicit winner selection:ProviderIDno longer flaps with replicated-map iteration order (freshest pong wins; newest registration is the no-pong fallback). Order-independence is asserted by running every table case forward and reversed.deriveStalenessThresholdpins the ping-config-to-freshness-window derivation so a config change cannot silently shrink the window; the former Redis-backed property tests are replaced by fast table tests, with the distributed plumbing behind theintegrationbuild tag.TestMainfails loudly when Redis cannot be provisioned instead of skipping every test and turning the lane vacuously green. CI runsmake itestso the lane has one source of truth.Runtime
CancellationReasonSessionEndedso the terminalRunCompletedevent carries canonical provenance even when engine cancellation never delivered (CancelRunrolls its provisional reason back in exactly that case). The workflow terminates through the existing cancellation path; engine cancellation is now an expedite-only optimization, closing the gap where in-flight runs outlived their session on engines without cancellation.policy.RetryReason/RetryHintare aliases of the planner types (no import cycle exists); the mirrored constants and the runtime's deep-cloning conversion are deleted, so retiring a reason is a single compiler-propagated edit.EventKey) is preserved through the hinting sink, and pending agent-tool child workflows are canceled at the parent finish-by deadline.Verification
go build ./...,golangci-lint(0 issues), full unit suite with-race(54 packages), Docker-backed registry integration suite, and the end-to-end scenario suite all pass.