feat: Add Gemini Realtime provider implementing IRealtimeClient/IRealtimeClientSession

[tarekgh]

Summary

Adds a Gemini Live API provider implementing the Microsoft.Extensions.AI Realtime abstractions (IRealtimeClient / IRealtimeClientSession), enabling real-time audio, text, and function-calling conversations with Gemini models through the standardized MEAI interface.

This PR also updates the repository to depend on the official Microsoft.Extensions.AI.Abstractions 10.4.1 NuGet package (replacing the private 10.5.0-dev builds).

What's Included

New Files

• GoogleGenAIRealtimeClient.cs — IRealtimeClient implementation that wraps a Google.GenAI.Client and creates realtime sessions via the Gemini Live API.
• GoogleGenAIRealtimeSession.cs — IRealtimeClientSession implementation that manages the WebSocket connection, audio buffering, message mapping, and function call orchestration.
• GoogleGenAIRealtimeTest.cs — 118 unit tests covering the full surface area.

Modified Files

• GoogleGenAIExtensions.cs — Added AsIRealtimeClient() extension method.
• Directory.Packages.props — Updated Microsoft.Extensions.AI.Abstractions from 10.5.0-dev → 10.4.1.
• Live.cs — Minor adjustment to expose AsyncSession for the realtime provider.
• All packages.lock.json files regenerated.

Features

• ✅ Audio streaming — Append/commit pattern with automatic frame splitting (32KB max), ActivityStart/ActivityEnd framing
• ✅ Voice Activity Detection (VAD) — Configurable server-side VAD or manual client-controlled boundaries
• ✅ Text conversations — Send text messages and receive text/audio responses
• ✅ Function calling — Full tool invocation support with the FunctionInvokingRealtimeSession middleware; tool responses are batched into a single SendToolResponseAsync call
• ✅ Transcription — Input and output audio transcription
• ✅ Thread-safe sends — SemaphoreSlim serializes all WebSocket sends, safe for concurrent middleware + caller usage
• ✅ Graceful disposal — Race-safe dispose with proper exception handling for in-flight operations

Usage Example

using Google.GenAI;
using Microsoft.Extensions.AI;

// Create the Gemini client and wrap it as an IRealtimeClient
var geminiClient = new Client(apiKey: "YOUR_API_KEY");
IRealtimeClient realtimeClient = new GoogleGenAIRealtimeClient(
    geminiClient, "gemini-2.5-flash-native-audio-preview-12-2025");

// Define a tool for function calling
AIFunction getWeather = AIFunctionFactory.Create(
    (string location) => location switch
    {
        "Seattle"       => $"The weather in {location} is rainy, 55°F",
        "New York"      => $"The weather in {location} is cloudy, 70°F",
        "San Francisco" => $"The weather in {location} is foggy, 60°F",
        _               => $"Sorry, I don't have weather data for {location}."
    },
    "GetWeather",
    "Gets the current weather for a given location");

// Wrap with middleware (function invocation, logging, OpenTelemetry)
var wrappedClient = new RealtimeClientBuilder(realtimeClient)
    .UseFunctionInvocation(configure: session =>
    {
        session.AdditionalTools = [getWeather];
        session.MaximumIterationsPerRequest = 10;
    })
    .UseLogging()
    .Build(serviceProvider);

// Configure session options
var sessionOptions = new RealtimeSessionOptions
{
    OutputModalities = ["audio"],
    Instructions = "You are a helpful assistant.",
    Voice = "Puck",
    TranscriptionOptions = new TranscriptionOptions(),
    Tools = [getWeather],
    VoiceActivityDetection = new VoiceActivityDetectionOptions
    {
        Enabled = true,
        AllowInterruption = true,
    },
};

// Create a session and start streaming
await using var session = await wrappedClient.CreateSessionAsync(sessionOptions);

// Start listening for server messages in the background
_ = Task.Run(async () =>
{
    await foreach (var message in session.GetStreamingResponseAsync(cancellationToken))
    {
        switch (message)
        {
            case OutputTextAudioRealtimeServerMessage audio
                when audio.Type == RealtimeServerMessageType.OutputAudioDelta:
                PlayAudio(audio.Audio);
                break;

            case OutputTextAudioRealtimeServerMessage text
                when text.Type == RealtimeServerMessageType.OutputTextDelta:
                Console.Write(text.Text);
                break;

            case InputAudioTranscriptionRealtimeServerMessage transcription:
                Console.WriteLine($"You said: {transcription.Transcription}");
                break;
        }
    }
});

// Send a text message (function calls are handled automatically by middleware)
var item = new RealtimeConversationItem(
    [new TextContent("What is the weather in New York?")],
    role: ChatRole.User);
await session.SendAsync(new CreateConversationItemRealtimeClientMessage(item: item));
await session.SendAsync(new CreateResponseRealtimeClientMessage());

Key Design Decisions

1. Tool response batching — The MEAI FunctionInvokingRealtimeSession middleware sends separate CreateConversationItem per function result. Gemini expects all results in one SendToolResponseAsync call. The provider buffers results and flushes them as a single batch when CreateResponse arrives.

2. TurnComplete suppression after tool responses — After SendToolResponseAsync, Gemini automatically continues generating. Sending client_content with turn_complete: true causes the server to close the WebSocket. The provider tracks this via _lastSendWasToolResponse and skips TurnComplete accordingly.

3. VAD handling — When VAD is disabled (default), the provider wraps audio commits with explicit ActivityStart/ActivityEnd framing. When enabled, the server handles speech boundary detection automatically.

4. Audio buffer cap — Audio appends are capped at 10 MB to prevent unbounded memory growth. Frames exceeding 32 KB are automatically split.

Test Coverage

118 unit tests covering:

• Client and session lifecycle (construction, disposal, idempotent dispose)
• All message types (audio, text, function calls, transcription, errors)
• Edge cases (null args, empty buffers, concurrent dispose, exception swallowing)
• Function call flow (single/multiple results, batching, flag reset after tool cycle)
• VAD modes (enabled, disabled, default)
• BuildLiveConnectConfig mapping (all option combinations)

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[tarekgh]

CC @stephentoub