Skip to content

Sessions: implement real OpenRouter provider support + voice-capable models (audio_turns is currently a no-op) #665

Description

@AbirAbbas

Follow-up from #654 / discussion.

The session transport matrix already declares openrouter: {audio_turns}, but audio_turns has no handler anywhere in the control plane — realtime-offer hard-gates to provider=openai+transport=webrtc and posts to api.openai.com/v1/realtime/calls. So today the matrix advertises an OpenRouter capability that does nothing.

This issue tracks making OpenRouter a real, working session provider, including its voice-capable models.

Why

  • Add AgentField realtime sessions for WebRTC voice ingress #654 ships a deliberately provider-agnostic session DX (provider is an explicit, required knob) but only OpenAI is actually wired. OpenRouter is the declared second provider; it should either work or not be advertised.
  • OpenRouter brokers many voice/realtime-capable models; supporting it makes AgentField sessions genuinely multi-provider rather than OpenAI-locked.

Scope / open design questions

  • Decide what audio_turns actually is (turn-based audio request/response) vs. whether OpenRouter exposes a realtime/WebRTC path, and implement the matching control-plane handler — not just a matrix entry.
  • Wire the provider boundary (OpenRouter API key server-side, same "client never holds the key" model as OpenAI).
  • Map session config (model, voice, modalities, tools) onto OpenRouter's API shape.
  • Surface OpenRouter's voice-capable models so @app.session(provider="openrouter", ...) can select them.

Acceptance criteria (behavior)

Related: #662 (sideband tools), #663 (VAD/turn-detection config), #664 (websocket transport + live loop scaffolding).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:aiAI/LLM integrationarea:control-planeControl plane server functionalityenhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions