TypeScript version: openai-realtime-api

[transitive-bullshit]

@khorwood-openai mentioned in another issue possibly replacing the current JS package with a TS version. I think most of the community would prefer that, since it's the only way to guarantee a solid DX and internal consistency, so I went ahead and took a stab at it.

See https://github.com/transitive-bullshit/openai-realtime-api

• This is a strongly-typed fork written in TypeScript.
• All classes, events, and methods are the same as in the JS version, so it's meant as a drop-in replacement, just with much stronger typing guarantees.
• My main contribution is that all events and handlers are typed via events.ts and RealtimeEventHandler.ts.
• The event types explicitly separate between server events, client events, and the 5 unofficial custom utility events.
• It also includes fixes for TypeScript error trying to consume package from node.js. EventHandlerCallbackType not defined #3, improve URL handling for custom queries passed via URL #11, Websocket handlers shouldn't throw exceptions #12, Manual tool calls don't work #14, Use it with Azure OpenAI #17, Bugfix/issue 27 invalid item access #29, Add example to make adoption easier. #34, fix: hard-coded url in in Node.js WebSocket connection #35, fix usage with cloudflare workers #37, Typed events #43, Fix onNext #44, and likely others.
• When porting to TS, there were lots of places where the strong typing resulted in small bug fixes that are not necessarily captured by current issues.
• There are a few places where I updated the code to use more modern TS conventions (e.g., one example is using structuredClone(data) instead of JSON.parse(JSON.stringify(data)))
• It includes several Node.js demos to make it easier to test the realtime API.
• It includes an imported version of OpenAI's Realtime Console, modified to replace @openai/realtime-api-beta with openai-realtime-api. It's a 99% drop-in replacement; I just had to tidy up some types since openai-realtime-api is a bit more strict.
• When I started out, I wasn't sure how 1:1 the TS port would be, so I created a new repo versus forking, but it shouldn't be too hard to compare between the two.

Very open to feedback && would love to merge any part of my TS port back upstream. Ideally once the official version of updated to use TS and stronger typing, I can deprecate my package.

cc @Stevenic and @ZJONSSON who both seem active on the TS side of things in addition to the rest of the OpenAI's team.

[Stevenic]

thanks for contributing this... will definitely take a look

[morrow-lw-dog]

Hi, @transitive-bullshit

I used your library and found a little problem:

The waitForNext method doesn't work, no matter what the event name is

const client = new RealtimeClient({
  url: "wss://my-relay-server-domain.com",
  sessionConfig: {
    turn_detection: {
      type: "server_vad",
    },
  },
});

// I made up the event name on a whim, so feel free to change it; none seem to take effect anyway.
client.waitForNext("session.update").then(ev => {
   // This line never works. I've checked by setting a breakpoint, 
   // and when dispatch method call, there's no event name registered in your eventHandlers.
   console.log("session.update", ev);
});

client.connect()

I have tried use client.realtime.waitForNext("session.update"), it worked.
So, Either remove the non-effective event names from the set of event names in the function signature of the client.waitForNext method, or make these events effective for client.waitForNext.