Interrupt the model with text message

[thoddnn]

Hello,

Would it be possible to add functionality to interrupt the model's audio output from webRTC template with a text message, similar to how it works in the playground ? (The interruption behavior with audio works great.)

Thank you!

[htaoufikallah]

try this from https://github.com/openai/openai-realtime-api-beta readme

// Send a item and triggers a generation
client.sendUserMessageContent([{ type: 'input_text', text: `How are you?` }]);

[thoddnn]

thanks @htaoufikallah but it doesn't work. Once the I get the response.done event from the server the audio doesn't stop.

I’ve tried using response.cancel, conversation.item.truncate then conversation.item.create with input_text and response.create but the audio continues playing until the end of the transcript.

(I'm using the webRTC template)

[sitozeta]

I need it too. A realtime model that works only with text data, pls if u know how do it reply this message. Ty

[thoddnn]

It works using the cancelResponse function from the WebSocket example: https://github.com/openai/openai-realtime-api-beta?tab=readme-ov-file#interrupting-the-model.

I think it is all about modifying the audioElement from the WebRTC template to make the audio interrupt and restart correctly.

[robclouth]

@thoddnn did you ever figure this out?

[prad-human-007]

Yes this is a problem in the realtime API system. I cannot stop the audio after the response.done has been sent.

[robclouth]

anyone figured out any hacks? like restarting the stream or something? commiting empty audio?

[thoddnn]

@thoddnn did you ever figure this out?

Nope 😅