FastRTC - Custom OpenAI Realtime AsyncStreamHandler - how to integrate existing OpenAI tools

[gsvitak-workwave]

I have a existing OpenAI Tool that works great for sending text to the OpenAI. I would like to integrate this existing tool into the FastRTC framework so I can use OpenAI Realtime API.

I have followed the cookbook and created my AsyncStreamHandler. I am just not sure how to integrate the OpenAI tool into the AsyncStreamHandler. For speed purposes, I would like to avoid an extra round trip of converting the voice to text via the RealTimeAPI and then making another OpenAI call with the text calling the OpenAI tool. I think It would great to pass the tool in the API call to the Open AI Realtime call from FastRTC.

I see in the OpenAI documentation, I can do this via the following code.

I am hopeful I can get some guidance on how to achieve via FastRTC flow.

Thank you in advance.

import { tool, RealtimeAgent } from '@openai/agents/realtime';
import { z } from 'zod';

const getWeather = tool({
  name: 'get_weather',
  description: 'Return the weather for a city.',
  parameters: z.object({ city: z.string() }),
  async execute({ city }) {
    return `The weather in ${city} is sunny.`;
  },
});

const weatherAgent = new RealtimeAgent({
  name: 'Weather assistant',
  instructions: 'Answer weather questions.',
  tools: [getWeather],
});