Skip to content

Sessions: implement websocket transport + wire RealtimeSession.input()/say() live loop (currently scaffolding) #664

Description

@AbirAbbas

Follow-up from #654.

Two pieces of the session DX are intentionally scaffolding in #654 and should be tracked so they aren't mistaken for complete:

  1. websocket transport is declared but not implemented. SUPPORTED_SESSION_TRANSPORTS lists openai: {webrtc, websocket}, but realtime-offer only handles webrtc (it errors on anything else). WebSocket is OpenAI's recommended transport for server-to-server realtime connections, so it's a reasonable next transport to actually wire.

  2. RealtimeSession.input() / say() are placeholders. session.input() raises RuntimeError("populated by the AgentField control plane transport adapter") and say() just appends to an in-memory outbox. The handler-facing live loop (model audio/event → session.input() → handler → session.say() back into the session) isn't wired end-to-end; the validated path today is SDP proxy + tool routing. The PR body itself points at an earlier live spike rather than an in-repo end-to-end test for this.

Acceptance criteria (behavior)

  • A session handler can await session.input() and receive a real SessionTurn populated from the live transport (not a RuntimeError).
  • await session.say(...) produces audio/text back into the live session the user hears, not just an outbox entry.
  • transport=websocket for provider=openai establishes a working realtime session (or is removed from the supported matrix until implemented, so the matrix doesn't advertise an unsupported pair).

This is a tracking issue for finishing the live handler loop; it does not block the SDP-proxy + tool-routing surface that #654 ships.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:aiAI/LLM integrationarea:control-planeControl plane server functionalityenhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions