Move to AI SDK for MCP tools

[nickytonline]

When the OpenAI Responses API came out this summer with support for MCP servers, I initially looked at leveraging the AI ADK for this, but they only supported creating an experimental MCP client for each remote MCP server you wanted to use which mean generating n MCP clients for n MCP servers, and after that awaiting them to get the tools.

Because of that I went with the native OpenAI responses API call (#8)

https://github.com/pomerium/mcp-app-demo/blob/main/src/routes/api/chat.ts#L147-L161

        answer = await client.responses.create({
          instructions: systemPrompt,
          model,
          tools,
          input: conversationHistory,
          stream: true,
          user: userId,
          ...(model.startsWith('o3') || model.startsWith('o4')
            ? {
                reasoning: {
                  summary: 'detailed',
                },
              }
            : {}),
        })

It looks like the AI SDK supports this now, see vercel/ai#10026 . Let's move back to the AI SDK for this.

Just need to verify it supports the user field, and if not, I could open an issue for that/PR it up.

[nickytonline]

AI SDK Migration Plan

After chatting with Claude Code, here's the game plan.

Executive Summary

This plan outlines how to migrate the MCP Chat Demo from direct OpenAI Responses API usage to leveraging the Vercel AI SDK abstractions. The goal is to maximize AI SDK usage and only go bespoke when absolutely necessary.

Current State Analysis

What You Have Now

1. Server (src/routes/api/chat.ts): Direct OpenAI client.responses.create() with manual stream transformation
2. Client (src/components/Chat.tsx): useChat hook from AI SDK (underutilized) + custom useStreamingChat hook (1000+ lines)
3. Streaming: Custom line-by-line parsing converting Responses API events to a StreamEvent union type

Key Discovery: AI SDK Beta Has MCP Support

@ai-sdk/openai@3.0.0-beta.71 includes openai.tools.mcp() with full support for:

• serverLabel - Label to identify the MCP server
• serverUrl - URL for the MCP server
• authorization - OAuth access token
• headers - Custom headers for MCP requests
• allowedTools - Tool filtering options

This is the key unlock - you can now use AI SDK's streamText() with native MCP tool support.

---

Migration Strategy

Phase 1: Upgrade Dependencies

Required Package Updates:

{
  "ai": "^5.0.104",
  "@ai-sdk/openai": "^3.0.0-beta.71",
  "zod": "^3.25.76"
}

Note: The beta version is required for openai.tools.mcp(). The stable @ai-sdk/openai@2.x does not include MCP support yet.

Risk Assessment: Using beta introduces potential instability. Monitor the AI SDK releases for when MCP tools hit stable.

---

Phase 2: Server-Side Migration (src/routes/api/chat.ts)

Goal: Replace direct OpenAI client with AI SDK's streamText() using openai.responses() model and openai.tools.mcp().

Current Implementation (Remove)

const client = new OpenAI()
const stream = await client.responses.create({
  instructions: systemPrompt,
  model,
  tools,  // Manual tool array construction
  input: conversationHistory,
  stream: true,
  // ...
})
// Manual ReadableStream transformation...

Target Implementation

import { streamText } from 'ai'
import { openai } from '@ai-sdk/openai'

const result = streamText({
  model: openai.responses(model),
  system: systemPrompt,
  messages,  // AI SDK handles message format
  tools: {
    // MCP servers as AI SDK tools
    ...Object.fromEntries(
      Object.entries(servers)
        .filter(([_, server]) => server.status === 'connected')
        .map(([id, server]) => [
          sanitizeServerLabel(server.name),
          openai.tools.mcp({
            serverLabel: sanitizeServerLabel(server.name),
            serverUrl: server.url,
            authorization: bearerToken,
            headers: { Authorization: `Bearer ${bearerToken}` },
          }),
        ])
    ),
    // Code interpreter (if enabled)
    ...(codeInterpreter && isCodeInterpreterSupported(model)
      ? { code_interpreter: openai.tools.codeInterpreter() }
      : {}),
    // Web search (if enabled)
    ...(webSearch && isWebSearchSupported(model)
      ? { web_search: openai.tools.webSearch() }
      : {}),
  },
  providerOptions: {
    openai: {
      user: userId,
      ...(model.startsWith('o3') || model.startsWith('o4')
        ? { reasoningSummary: 'detailed' }
        : {}),
    },
  },
})

return result.toDataStreamResponse()

What This Gives You:

• No manual stream transformation
• AI SDK handles all wire format details
• Native tool result handling
• Proper error propagation
• Built-in usage token tracking

---

Phase 3: Client-Side Simplification

Goal: Remove or dramatically simplify useStreamingChat.ts by leveraging useChat callbacks.

Option A: Full AI SDK (Recommended if callbacks work)

// In Chat.tsx
const { messages, isLoading, append, stop } = useChat({
  body: chatBody,
  onToolCall: async ({ toolCall }) => {
    // AI SDK automatically handles MCP tool execution
    // You just need to render the tool call UI
    addToolEvent(toolCall)
  },
  experimental_onChunk: (chunk) => {
    // Handle special chunks (reasoning, code interpreter output)
    if (chunk.type === 'reasoning') {
      addReasoningEvent(chunk)
    }
  },
})

Option B: Hybrid (If some events need custom handling)

Keep a minimal version of useStreamingChat that only handles events the AI SDK doesn't expose properly:

// Reduced useStreamingChat.ts (~200 lines instead of 1000+)
export function useStreamingChat() {
  // Only handle:
  // 1. Code interpreter file annotations (if AI SDK doesn't expose them)
  // 2. Web search intermediate states (if AI SDK doesn't expose them)
  // 3. Reasoning accumulation (if experimental_onChunk doesn't work)
}

---

Phase 4: Event Type Rationalization

Current: 8 custom StreamEvent types with complex union

Target: Map AI SDK chunk types to your UI needs

AI SDK Chunk Type	Your UI Component	Action
text-delta	BotMessage	Use messages from useChat directly
tool-call	ToolCallMessage	Use onToolCall callback
tool-result	ToolCallMessage	Automatic via AI SDK
reasoning	ReasoningMessage	Use experimental_onChunk
error	BotError	Use onError callback

Custom handling still needed for:

• CodeInterpreterMessage - AI SDK's openai.tools.codeInterpreter() output format
• WebSearchMessage - AI SDK's openai.tools.webSearch() result format
• File annotations from code interpreter

---

Phase 5: Verify user Field Support

Per your issue #168, verify the user field works:

providerOptions: {
  openai: {
    user: userId,  // Should work per AI SDK docs
  },
}

The AI SDK documentation confirms user is supported:

"A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse"

---

What Stays Bespoke (And Why)

Feature	Why Custom	Lines Estimate
StreamEvent types	Type safety for your specific UI components	~50
Code interpreter UI mapping	AI SDK returns raw output, you need structured display	~100
File annotation handling	Download URLs, image display logic	~50
Web search result formatting	Your specific UI requirements	~50

Total custom code: ~250 lines (down from 1000+)

---

Implementation Checklist

Step 1: Dependencies

• Update package.json with new versions
• Run npm install
• Verify no peer dependency conflicts
• Update Zod if needed (requires ^3.25.76)

Step 2: Server Migration

• Import streamText from ai and openai from @ai-sdk/openai
• Replace new OpenAI() with openai.responses(model)
• Replace manual tool array with openai.tools.mcp() calls
• Replace openai.tools.codeInterpreter() and openai.tools.webSearch()
• Add providerOptions.openai.user for user tracking
• Replace manual stream transformation with result.toDataStreamResponse()
• Test all tool types work correctly

Step 3: Client Simplification

• Add onToolCall callback to useChat
• Add experimental_onChunk for reasoning/special events
• Test if code interpreter events come through properly
• Test if web search events come through properly
• Identify what useStreamingChat logic is still needed
• Refactor/remove unnecessary custom streaming code

Step 4: UI Component Updates

• Update ToolCallMessage to work with AI SDK tool call format
• Update CodeInterpreterMessage for new event structure
• Update WebSearchMessage for new event structure
• Verify BotMessage works with AI SDK messages
• Test ReasoningMessage with experimental chunks

Step 5: Testing & Validation

• Test MCP server connections
• Test code interpreter execution and file outputs
• Test web search with result display
• Test reasoning models (o3/o4)
• Test error handling
• Test streaming interruption/cancellation
• Verify user field is sent to OpenAI

---

Risks & Mitigations

Risk	Mitigation
Beta package instability	Pin exact version, monitor releases, have rollback plan
AI SDK chunk format changes	Abstract chunk handling, add type guards
Missing events in callbacks	Keep minimal custom stream handling as fallback
Zod upgrade breaks schemas	Test all schemas, update if needed

---

Resources

• AI SDK Docs
• OpenAI Provider Docs
• AI SDK MCP Tools
• PR #10026 - MCP Support
• Your Issue #168

---

Decision Point

Before starting implementation, verify:

1. Is @ai-sdk/openai@3.0.0-beta.71 stable enough for your use case?

   • Test in a branch first
   • Check GitHub issues for known problems

2. Does experimental_onChunk expose all events you need?

   • If not, keep minimal custom handling

3. Are you comfortable with Zod ^3.25.76 requirement?

   • May require schema updates

---

Generated: 2025-11-27