Skip to content

[ENG-1632] Audio turns in realtime session view#3734

Merged
connortbot merged 6 commits intomainfrom
connor/eng-1632
May 12, 2025
Merged

[ENG-1632] Audio turns in realtime session view#3734
connortbot merged 6 commits intomainfrom
connor/eng-1632

Conversation

@connortbot
Copy link
Collaborator

@connortbot connortbot commented May 10, 2025

Screenshot 2025-05-09 at 9 15 55 PM Screenshot 2025-05-09 at 9 53 28 PM

the way I chose start and ending events for different types was a little arbitrary, so we should make it clear the start and end events in the UI [DONE]

@vercel
Copy link

vercel bot commented May 10, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
helicone ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 12, 2025 8:45am
helicone-bifrost ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 12, 2025 8:45am
helicone-eu ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 12, 2025 8:45am

@promptless
Copy link
Contributor

promptless bot commented May 10, 2025

📝 Documentation updates detected! You can review documentation updates here

@fumedev
Copy link

fumedev bot commented May 10, 2025

Summary

  • Introduces real-time audio message handling in the session view.
  • Enhances user interface to clearly indicate audio start and end events.
  • Improves robustness by handling cases where an end event might be missed.
  • Combines multiple audio segments into single messages for both user and assistant audio.
  • Adjusts timestamp handling to ensure accurate chronological order of messages.

const timeA = a.timestamp ? new Date(a.timestamp).getTime() : 0;
const timeB = b.timestamp ? new Date(b.timestamp).getTime() : 0;
return timeA - timeB;
return (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR implements comprehensive audio turn tracking in the realtime session view by adding event lifecycle tracking and improving audio buffer handling across the codebase.

  • Added new Message fields trigger_event_id and ending_event_id across multiple type definitions to explicitly track audio message lifecycles
  • Implemented new 'response.audio.combined' message type and refactored audio buffer combination logic into a reusable function
  • Updated timestamp sorting to prioritize start_timestamp with fallback to timestamp for more accurate event sequencing
  • Added UI element in Realtime component showing event ID flow with -> arrows for clearer visualization
  • Removed 1ms gap requirement between steps and improved path generation for different message types

21 file(s) reviewed, 12 comment(s)
Edit PR Review Bot Settings | Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants