Skip to content

Commit 7e72b4a

Browse files
authored
feat: Multi-turn conversation state management with type safety (#124)
## Summary - Add multi-turn conversation state management with `StateAccessor` pattern for pluggable persistence - Add approval workflow for tool execution (`requireApproval`, `approveToolCalls`, `rejectToolCalls`) - Improve type safety by replacing type casts with type guards - Refactor `executeToolsIfNeeded` from ~400 lines to ~110 lines by extracting 14 focused helper methods ## Key Changes ### New State Management Features - `StateAccessor` interface for pluggable state persistence (in-memory, database, etc.) - `ConversationState` type for tracking conversation status, messages, pending approvals - `createInitialState()`, `updateState()` utilities for state management - Support for interruption handling and resumption ### Approval Workflow - Tools can specify `requireApproval: true` to pause execution - Call-level `requireApproval` function for dynamic approval logic - `approveToolCalls` and `rejectToolCalls` arrays to resume with decisions - `getPendingToolCalls()` and `requiresApproval()` methods on `ModelResult` ### Type Safety Improvements - Type guards (`isValidUnsentToolResult`, `isValidParsedToolCall`) instead of type casts - Made `resolveAsyncFunctions` generic to eliminate double-casting - `isEventStream` type guard handles both streaming and non-streaming responses ## Examples ### Basic State Management ```typescript import { callModel, type ConversationState, type StateAccessor } from '@openrouter/sdk'; // In-memory state storage (use database in production) const conversations = new Map<string, ConversationState>(); const stateAccessor: StateAccessor = { load: async () => conversations.get('conv-123') ?? null, save: async (state) => { conversations.set('conv-123', state); }, }; const result = client.callModel({ model: 'anthropic/claude-sonnet-4', input: [{ type: 'text', text: 'Hello!' }], state: stateAccessor, }); const text = await result.getText(); const state = await result.getState(); // Access conversation state ``` ### Tool with Approval Required ```typescript import { tool } from '@openrouter/sdk'; const deleteFileTool = tool({ name: 'delete_file', description: 'Delete a file from the filesystem', inputSchema: z.object({ path: z.string().describe('File path to delete'), }), outputSchema: z.object({ success: z.boolean() }), requireApproval: true, // Pause for human approval execute: async ({ path }) => { await fs.unlink(path); return { success: true }; }, }); const result = client.callModel({ model: 'anthropic/claude-sonnet-4', input: [{ type: 'text', text: 'Delete the temp.txt file' }], tools: [deleteFileTool], state: stateAccessor, }); // Check if approval is needed if (await result.requiresApproval()) { const pending = await result.getPendingToolCalls(); console.log('Pending approval:', pending); // Show to user, get decision... } ``` ### Resume with Approval Decision ```typescript // After user approves/rejects... const result = client.callModel({ model: 'anthropic/claude-sonnet-4', input: [], // Empty - resuming from state tools: [deleteFileTool], state: stateAccessor, approveToolCalls: ['call_abc123'], // Approve specific tool calls rejectToolCalls: ['call_def456'], // Reject others }); const text = await result.getText(); // Continues execution ``` ### Dynamic Approval Logic ```typescript const result = client.callModel({ model: 'anthropic/claude-sonnet-4', input: [{ type: 'text', text: 'Process these files' }], tools: [readFileTool, writeFileTool, deleteFileTool], state: stateAccessor, // Dynamic approval based on tool and context requireApproval: (toolCall, context) => { // Always approve read operations if (toolCall.name === 'read_file') return false; // Require approval for destructive operations if (toolCall.name === 'delete_file') return true; // Require approval after 3 turns return context.numberOfTurns > 3; }, }); ``` ### Stop Conditions ```typescript import { stepCountIs, hasToolCall, maxCost } from '@openrouter/sdk'; const result = client.callModel({ model: 'anthropic/claude-sonnet-4', input: [{ type: 'text', text: 'Research this topic thoroughly' }], tools: [searchTool, summarizeTool], // Stop when any condition is met stopWhen: [ stepCountIs(10), // Max 10 tool execution rounds hasToolCall('summarize'), // Stop when summarize is called maxCost(0.50), // Budget limit ], }); ``` ## Test plan - [x] All 320 existing tests pass - [x] New unit tests for conversation state utilities (21 tests) - [x] New E2E tests for state management integration (5 tests) - [x] Build passes with no TypeScript errors - [x] Lint passes with no warnings
1 parent 1e49648 commit 7e72b4a

File tree

9 files changed

+1989
-233
lines changed

9 files changed

+1989
-233
lines changed

src/funcs/call-model.ts

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,16 @@ export function callModel<TTools extends readonly Tool[]>(
124124
request: CallModelInput<TTools>,
125125
options?: RequestOptions,
126126
): ModelResult<TTools> {
127-
const { tools, stopWhen, ...apiRequest } = request;
127+
// Destructure state management options along with tools and stopWhen
128+
const {
129+
tools,
130+
stopWhen,
131+
state,
132+
requireApproval,
133+
approveToolCalls,
134+
rejectToolCalls,
135+
...apiRequest
136+
} = request;
128137

129138
// Convert tools to API format - no cast needed now that convertToolsToAPIFormat accepts readonly
130139
const apiTools = tools ? convertToolsToAPIFormat(tools) : undefined;
@@ -144,10 +153,12 @@ export function callModel<TTools extends readonly Tool[]>(
144153
client,
145154
request: finalRequest,
146155
options: options ?? {},
147-
// Preserve the exact TTools type instead of widening to Tool[]
148-
tools: tools as TTools | undefined,
149-
...(stopWhen !== undefined && {
150-
stopWhen,
151-
}),
156+
tools,
157+
...(stopWhen !== undefined && { stopWhen }),
158+
// Pass state management options
159+
...(state !== undefined && { state }),
160+
...(requireApproval !== undefined && { requireApproval }),
161+
...(approveToolCalls !== undefined && { approveToolCalls }),
162+
...(rejectToolCalls !== undefined && { rejectToolCalls }),
152163
} as GetResponseOptions<TTools>);
153164
}

src/index.ts

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,14 +5,18 @@
55
// Async params support
66
export type {
77
CallModelInput,
8+
CallModelInputWithState,
89
FieldOrAsyncFunction,
910
ResolvedCallModelInput,
1011
} from './lib/async-params.js';
1112
export type { Fetcher, HTTPClientOptions } from './lib/http.js';
1213
// Tool types
1314
export type {
1415
ChatStreamEvent,
16+
ConversationState,
17+
ConversationStatus,
1518
ResponseStreamEvent as EnhancedResponseStreamEvent,
19+
HasApprovalTools,
1620
InferToolEvent,
1721
InferToolEventsUnion,
1822
InferToolInput,
@@ -21,19 +25,24 @@ export type {
2125
NextTurnParamsContext,
2226
NextTurnParamsFunctions,
2327
ParsedToolCall,
28+
PartialResponse,
29+
StateAccessor,
2430
StepResult,
2531
StopCondition,
2632
StopWhen,
2733
Tool,
34+
ToolApprovalCheck,
2835
ToolExecutionResult,
2936
ToolExecutionResultUnion,
37+
ToolHasApproval,
3038
ToolPreliminaryResultEvent,
3139
ToolStreamEvent,
3240
ToolWithExecute,
3341
ToolWithGenerator,
3442
TurnContext,
3543
TypedToolCall,
3644
TypedToolCallUnion,
45+
UnsentToolResult,
3746
Warning,
3847
} from './lib/tool-types.js';
3948
export type { BuildTurnContextOptions } from './lib/turn-context.js';
@@ -101,14 +110,27 @@ export {
101110
// Tool creation helpers
102111
export { tool } from './lib/tool.js';
103112
export {
113+
hasApprovalRequiredTools,
104114
hasExecuteFunction,
105115
isGeneratorTool,
106116
isRegularExecuteTool,
107117
isToolPreliminaryResultEvent,
118+
toolHasApprovalConfigured,
108119
ToolType,
109120
} from './lib/tool-types.js';
110121
// Turn context helpers
111122
export { buildTurnContext, normalizeInputToArray } from './lib/turn-context.js';
123+
// Conversation state helpers
124+
export {
125+
appendToMessages,
126+
createInitialState,
127+
createRejectedResult,
128+
createUnsentResult,
129+
generateConversationId,
130+
partitionToolCalls,
131+
toolRequiresApproval,
132+
updateState,
133+
} from './lib/conversation-state.js';
112134
// Real-time tool event broadcasting
113135
export { ToolEventBroadcaster } from './lib/tool-event-broadcaster.js';
114136
export * from './sdk/sdk.js';

src/lib/async-params.ts

Lines changed: 58 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
import type * as models from '../models/index.js';
2-
import type { StopWhen, Tool, TurnContext } from './tool-types.js';
2+
import type { ParsedToolCall, StateAccessor, StopWhen, Tool, TurnContext } from './tool-types.js';
3+
4+
// Re-export Tool type for convenience
5+
export type { Tool } from './tool-types.js';
36

47
/**
58
* Type guard to check if a value is a parameter function
@@ -29,19 +32,67 @@ function buildResolvedRequest(
2932
export type FieldOrAsyncFunction<T> = T | ((context: TurnContext) => T | Promise<T>);
3033

3134
/**
32-
* Input type for callModel function
33-
* Each field can independently be a static value or a function that computes the value
34-
* Generic over TTools to enable proper type inference for stopWhen conditions
35+
* Base input type for callModel without approval-related fields
3536
*/
36-
export type CallModelInput<TTools extends readonly Tool[] = readonly Tool[]> = {
37+
type BaseCallModelInput<TTools extends readonly Tool[] = readonly Tool[]> = {
3738
[K in keyof Omit<models.OpenResponsesRequest, 'stream' | 'tools'>]?: FieldOrAsyncFunction<
3839
models.OpenResponsesRequest[K]
3940
>;
4041
} & {
4142
tools?: TTools;
4243
stopWhen?: StopWhen<TTools>;
44+
/**
45+
* Call-level approval check - overrides tool-level requireApproval setting
46+
* Receives the tool call and turn context, can be sync or async
47+
*/
48+
requireApproval?: (
49+
toolCall: ParsedToolCall<TTools[number]>,
50+
context: TurnContext
51+
) => boolean | Promise<boolean>;
52+
};
53+
54+
/**
55+
* Approval params when state is provided (allows approve/reject)
56+
*/
57+
type ApprovalParamsWithState<TTools extends readonly Tool[] = readonly Tool[]> = {
58+
/** State accessor for multi-turn persistence and approval gates */
59+
state: StateAccessor<TTools>;
60+
/** Tool call IDs to approve (for resuming from awaiting_approval status) */
61+
approveToolCalls?: string[];
62+
/** Tool call IDs to reject (for resuming from awaiting_approval status) */
63+
rejectToolCalls?: string[];
64+
};
65+
66+
/**
67+
* Approval params when state is NOT provided (forbids approve/reject)
68+
*/
69+
type ApprovalParamsWithoutState = {
70+
/** State accessor for multi-turn persistence and approval gates */
71+
state?: undefined;
72+
/** Not allowed without state - will cause type error */
73+
approveToolCalls?: never;
74+
/** Not allowed without state - will cause type error */
75+
rejectToolCalls?: never;
4376
};
4477

78+
/**
79+
* Input type for callModel function
80+
* Each field can independently be a static value or a function that computes the value
81+
* Generic over TTools to enable proper type inference for stopWhen conditions
82+
*
83+
* Type enforcement:
84+
* - `approveToolCalls` and `rejectToolCalls` are only valid when `state` is provided
85+
* - Using these without `state` will cause a TypeScript error
86+
*/
87+
export type CallModelInput<TTools extends readonly Tool[] = readonly Tool[]> =
88+
BaseCallModelInput<TTools> & (ApprovalParamsWithState<TTools> | ApprovalParamsWithoutState);
89+
90+
/**
91+
* CallModelInput variant that requires state - use when approval workflows are needed
92+
*/
93+
export type CallModelInputWithState<TTools extends readonly Tool[] = readonly Tool[]> =
94+
BaseCallModelInput<TTools> & ApprovalParamsWithState<TTools>;
95+
4596
/**
4697
* Resolved CallModelInput (all functions evaluated to values)
4798
* This is the type after all async functions have been resolved to their values
@@ -70,8 +121,8 @@ export type ResolvedCallModelInput = Omit<models.OpenResponsesRequest, 'stream'
70121
* // resolved.temperature === 0.2
71122
* ```
72123
*/
73-
export async function resolveAsyncFunctions(
74-
input: CallModelInput,
124+
export async function resolveAsyncFunctions<TTools extends readonly Tool[] = readonly Tool[]>(
125+
input: CallModelInput<TTools>,
75126
context: TurnContext,
76127
): Promise<ResolvedCallModelInput> {
77128
// Build array of resolved entries

0 commit comments

Comments
 (0)