Model Is Having Trouble Understanding

[alfiesal]

Hi,

I’ve been testing this tool in conjunction with the Twilio integration and noticed that the model has significant difficulty understanding what is being said to it.

However, when I use the OpenAI Playground, I don’t encounter this issue. There, the model accurately comprehends about 99% of what is spoken.

[Evanc123]

+1, it seems to drop ~20-40% of my sentences compared to the playground. I note that another twilio/openAI demo repo also reports a similar problem: twilio-samples/speech-assistant-openai-realtime-api-node#31

they speculate that:

Given that the performance difference is so pronounced, I suspect the issue is related to the audio quality or transmission when using Twilio’s phone integration. It seems like the model struggles to correctly interpret spoken words despite them being understood perfectly in other environments.

[alfiesal]

I also tested Eleven Labs in combination with Twilio, and there are no problems there.

[da1z]

seems like issue is on openai realtime api side. the transcript returned from api is correct but response seems to be to something else

[jeremym148]

i experience the same gap between webrtc web app vs voice agent couple to twillio

with twilio the quality is dropping 20-40% also

someone found the root cause or the solution?

[elibenyosef]

Having the same problem here