Skip to content

Canonical model transcripts: exact token counting, tool naming, health, and session lifecycle#185

Merged
raphael merged 9 commits into
mainfrom
fix/canonical-model-transcripts
Jul 18, 2026
Merged

Canonical model transcripts: exact token counting, tool naming, health, and session lifecycle#185
raphael merged 9 commits into
mainfrom
fix/canonical-model-transcripts

Conversation

@raphael

@raphael raphael commented Jul 18, 2026

Copy link
Copy Markdown
Member

Canonical model transcripts end to end: exact Anthropic token counting, one provider tool-name projection, deterministic registry health, and durable session-lifecycle enforcement in the run loop.

Model providers

  • Anthropic exact token counting: the adapter now separates the canonical request encoding (messages, tools, cache markers, tool choice, betas) from completion policy (max_tokens, temperature, thinking). CountTokens consumes the encoding directly, so counting works without a default MaxTokens and can never drift from the completion encoding ("tools": [] vs omitted).
  • Tool-examples beta: attached additively (WithHeaderAdd) so caller-configured betas (e.g. 1M context) survive, and only when a tool authors examples — header-compatible gateways reject unknown beta identifiers. Live-verified against Vertex rawPredict (usage.input_tokens deltas): Claude-on-Vertex delivers input_examples natively with no beta activation and ignores the header, so no transport gate exists.
  • Counting projection owned once: model.CountingRequest is the single owner of the TokenCounter projection (thinking stripped, thinking-only messages dropped); anthropic, bedrock, and vertex all consume it, deleting vertex's divergent private copy.
  • Canonical tool names: features/model/toolname owns the provider-name projection for the shared [a-zA-Z0-9_-]{1,64} contract (Claude Messages API + OpenAI Responses); openai's byte-identical copy is deleted and toolname.BuildMaps owns the injectivity invariant for anthropic, bedrock, and openai. Direct Anthropic now sends fully namespaced tool names (leaf-stripping removed); provider names are per-request projections, never persisted.

Registry

  • computeToolsetHealth is a pure function with explicit winner selection: ProviderID no longer flaps with replicated-map iteration order (freshest pong wins; newest registration is the no-pong fallback). Order-independence is asserted by running every table case forward and reversed.
  • deriveStalenessThreshold pins the ping-config-to-freshness-window derivation so a config change cannot silently shrink the window; the former Redis-backed property tests are replaced by fast table tests, with the distributed plumbing behind the integration build tag.
  • The integration tag is an explicit opt-in: TestMain fails loudly when Redis cannot be provisioned instead of skipping every test and turning the lane vacuously green. CI runs make itest so the lane has one source of truth.

Runtime

  • Session lifecycle enforced at turn boundaries: every planner activity (start and resume) consults the durable session status first and refuses to plan under an ended session, recording CancellationReasonSessionEnded so the terminal RunCompleted event carries canonical provenance even when engine cancellation never delivered (CancelRun rolls its provisional reason back in exactly that case). The workflow terminates through the existing cancellation path; engine cancellation is now an expedite-only optimization, closing the gap where in-flight runs outlived their session on engines without cancellation.
  • Timeout is terminal: tool timeouts and exhausted budgets no longer produce planner retry hints — elapsed execution time is not an instruction to repeat a call.
  • One retry vocabulary: policy.RetryReason/RetryHint are aliases of the planner types (no import cycle exists); the mirrored constants and the runtime's deep-cloning conversion are deleted, so retiring a reason is a single compiler-propagated edit.
  • Stream event identity (EventKey) is preserved through the hinting sink, and pending agent-tool child workflows are canceled at the parent finish-by deadline.

Verification

  • go build ./..., golangci-lint (0 issues), full unit suite with -race (54 packages), Docker-backed registry integration suite, and the end-to-end scenario suite all pass.
  • Vertex beta behavior verified empirically against the live rawPredict endpoint.

raphael added 9 commits July 13, 2026 01:20
Give transport boundaries canonical metadata codecs and let Bedrock replay provider-authored citation blocks without flattening or losing source locations.
Remove the open-ended metadata side channel so clean EOF exposes canonical content, usage, and stop reason through one typed response contract.
Honor ToolChoiceNone only when no tools are advertised so callers never mistake Bedrock auto-selection for a no-tools guarantee.
Describe the invariant in terms of forbidden budgeted work so provider history catalogs and terminal bookkeeping calls remain explicit.
Decode Bedrock's canonical over-context CountTokens response as the exact measurement so history compression can run without heuristic estimates.
Split the Anthropic adapter into a canonical request encoding shared by
Complete, Stream, and CountTokens plus a completion-policy layer, so token
counting no longer inherits completion-only requirements (max_tokens) and
cannot drift from the completion encoding (empty tools serialization).

Attach the tool-examples beta header additively (WithHeaderAdd) so
caller-configured betas survive, and only when authored examples are
present. Live-verified against Vertex rawPredict: Claude-on-Vertex delivers
input_examples natively with no beta activation and ignores the header, so
no transport gate is needed.

Own the token-counting projection once as model.CountingRequest; the
anthropic, bedrock, and vertex counters all consume it, deleting vertex's
divergent private copy. Make features/model/toolname the canonical
provider-name projection for the shared [a-zA-Z0-9_-]{1,64} contract:
openai delegates to it and toolname.BuildMaps owns the injectivity
invariant for anthropic, bedrock, and openai.
…tion lane

Extract computeToolsetHealth as the pure owner of the staleness rule with
explicit winner selection, so ProviderID no longer depends on replicated-map
iteration order, and pin the staleness-threshold derivation with
deriveStalenessThreshold so a config change cannot silently shrink the
window providers have to respond. Fast table tests cover the rule; the
Redis-backed tests move behind the integration build tag.

The integration tag is an explicit opt-in to Docker-backed tests, so
TestMain now fails loudly when Redis cannot be provisioned instead of
skipping every test and turning the lane vacuously green. CI invokes make
itest so the integration lane has one source of truth.
Make the durable session status the authority that stops in-flight runs:
every planner activity (start and resume) consults it first and refuses to
plan under an ended session, recording CancellationReasonSessionEnded so
the terminal RunCompleted event carries canonical provenance even when
engine cancellation never delivered. The workflow maps the refusal through
the existing cancellation terminal path, demoting engine cancellation to an
expedite-only optimization.

Remove the timeout retry hint: elapsed execution time is not an instruction
to repeat a call, so timeout and exhausted-budget failures are terminal for
the current run. Collapse the policy retry vocabulary onto the planner
enum via type aliases, deleting the mirrored constants and the runtime's
deep-cloning conversion. Preserve stream event identity through the hinting
sink, and cancel pending agent-tool child workflows at the parent
finish-by deadline.
@raphael
raphael merged commit 7b615c9 into main Jul 18, 2026
2 checks passed
@raphael
raphael deleted the fix/canonical-model-transcripts branch July 18, 2026 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant