Skip to content

Conversation

@ellis-driscoll
Copy link
Contributor

@ellis-driscoll ellis-driscoll commented Jan 5, 2026

Background

When using OpenAI's conversation mode (providerOptions.openai.conversation), the provider persists conversation history server-side. The AI SDK's tool loop was re-sending all prior messages (including assistant items the model had already produced) on subsequent iterations, causing duplicate item errors and misaligned tool outputs.

Summary

This change implements delta prompting for OpenAI conversation mode in both generateText and streamText:

  • On the first model call, send the full prompt
  • On subsequent loop iterations, send only new client-originated items (tool results)
  • Fall back to full messages when the delta is empty (e.g., when a provider-executed tool has deferred results but no client tool results exist yet)

Added getOpenAIConversationId helper to detect when conversation mode is active. The helper returns undefined for empty strings to avoid accidental activation.

Manual Verification

Using the Responses API with a tool that auto-executes. Without this fix, the second call would fail with a duplicate item error from OpenAI. Running this script from packages/ai will throw an error on main but works with this change.

import { generateText, stepCountIs } from './src/index.js';
import { createOpenAI } from '../openai/src/index.js';
import { tool } from '../provider-utils/src/index.js';
import { z } from 'zod/v4';

const OPENAI_API_KEY = 'sk-svcacct-...';
const openai = createOpenAI({ apiKey: OPENAI_API_KEY });

async function main() {
  const result = await generateText({
    model: openai.responses('gpt-4o-mini'),
    prompt: 'Get the secret from the tool call and tell me what it is',
    stopWhen: stepCountIs(40),
    tools: {
      secret: tool({
        description: 'Returns the secret for any input string',
        inputSchema: z.object({
          input: z.string(),
        }),
        execute: async ({ input }) => {
          return `purple monkey dishwasher`;
        },
      }),
    },
    providerOptions: {
      openai: { conversation: 'conv_...' },
    },
  });

  console.log(result.text);
}

main();

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant