Skip to content

think: expose the admitted turn's requestId on TurnContext for beforeTurn #2240

Description

@cogell

beforePersist (PersistContext.requestId), onChatResponse (ChatResponseResult.requestId) and onChatError (ChatErrorContext.requestId) all identify the chat request they run for, but beforeTurn's TurnContext does not. Anything a subclass starts in beforePersist for a request (a trace span, a timer, a ledger entry) cannot be matched to the turn that later runs for it, short of inferring the link from the message list, which breaks as soon as two requests overlap in the pre-persist window.

The id is already in scope where Think builds the hook's argument: every admitted turn runs inside admittedTurnContext (packages/think/src/think.ts), and _runInferenceLoop assembles ctx right before this.beforeTurn(ctx):

const ctx: TurnContext = {
  system,
  messages,
  tools,
  model,
  continuation: input.continuation,
  body: input.body
};

Proposed change: one field.

const ctx: TurnContext = {
  // ...as today
  body: input.body,
  requestId: admittedTurnContext.getStore()?.requestId
};
export interface TurnContext {
  // ...
  /**
   * The chat request this turn runs for — the same id `beforePersist`,
   * `onChatResponse` and `onChatError` see. Undefined outside an admitted
   * turn (entry points that bypass admission).
   */
  requestId?: string;
}

Optional, so programmatic entry points that bypass admission stay valid and no existing subclass changes.

Context: we open an OpenTelemetry span per turn in beforePersist (it is the first thing a message pays for, and a refused charge is a turn that never ran) and need beforeTurn to adopt that request's span. We run exactly this as a patch on @cloudflare/think@0.17.0 today. Happy to open the PR if the shape is acceptable.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions