Skip to content

Sessions: implement real OpenRouter provider support + voice-capable models (audio_turns is currently a no-op) #665

Description

@AbirAbbas

Follow-up from #654 / discussion.

The session transport matrix already declares openrouter: {audio_turns}, but audio_turns has no handler anywhere in the control plane — realtime-offer hard-gates to provider=openai+transport=webrtc and posts to api.openai.com/v1/realtime/calls. So today the matrix advertises an OpenRouter capability that does nothing.

This issue tracks making OpenRouter a real, working session provider, including its voice-capable models.

Why

  • Add AgentField realtime sessions for WebRTC voice ingress #654 ships a deliberately provider-agnostic session DX (provider is an explicit, required knob) but only OpenAI is actually wired. OpenRouter is the declared second provider; it should either work or not be advertised.
  • OpenRouter brokers many voice/realtime-capable models; supporting it makes AgentField sessions genuinely multi-provider rather than OpenAI-locked.

Scope / open design questions

  • Decide what audio_turns actually is (turn-based audio request/response) vs. whether OpenRouter exposes a realtime/WebRTC path, and implement the matching control-plane handler — not just a matrix entry.
  • Wire the provider boundary (OpenRouter API key server-side, same "client never holds the key" model as OpenAI).
  • Map session config (model, voice, modalities, tools) onto OpenRouter's API shape.
  • Surface OpenRouter's voice-capable models so @app.session(provider="openrouter", ...) can select them.

Acceptance criteria (behavior)

Related: #662 (sideband tools), #663 (VAD/turn-detection config), #664 (websocket transport + live loop scaffolding).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:aiAI/LLM integrationarea:control-planeControl plane server functionalityenhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions