Gemini 3.1 Flash Live Preview - STT, VAD, Silence Duration, and Tool Calling Issues

[AryanKarumuri]

Description of the bug:

We are observing multiple issues while using Gemini 3.1 Flash Live Preview in a voice-based application.

Environment

• Model: Gemini 3.1 Flash Live Preview
• Audio input/output enabled
• Automatic Activity Detection (VAD) enabled

Configuration:

setup: {
  model: `models/${GEMINI_LIVE_MODEL}`,
  generationConfig,
  systemInstruction: {
    parts: [{ text: effectivePromptText }],
  },
  realtimeInputConfig: {
    automaticActivityDetection: {
      disabled: false,
      silenceDurationMs,
      prefixPaddingMs,
    },
    turnCoverage: 'TURN_INCLUDES_ONLY_ACTIVITY',
  },
  inputAudioTranscription: {},
  outputAudioTranscription: {},
  sessionResumption,
  tools: [TOPIC_TRANSITION_TOOL, END_CALL_TOOL],
}

silenceDurationMs = 6000;

Issues Observed

1. STT Accuracy Issues

Example:

User speech:

I will identify the odd one out

Transcription:

Old one out

The transcription drops multiple words and changes the meaning of the utterance.

2. silenceDurationMs Not Being Honored

We configured:

silenceDurationMs = 6000;

However, the model appears to end the user turn after approximately 2000ms of silence, which seems to match the default behavior.

We verified this by:

• Measuring actual silence duration before turn completion
• Checking conversation logs
• Confirming that logs show the configured value of 6000ms

Despite this, the turn closes after roughly 2000ms of silence.

3. VAD Cuts Off Users Mid-Sentence

The VAD occasionally detects end-of-turn while the user is still speaking.

This results in:

• Partial utterances being processed
• Responses generated before the user finishes speaking
• Interrupted conversational flow

4. Inconsistent Tool Calling

Tool invocation appears unreliable.

Even when:

• Tool descriptions are explicit
• Usage instructions are clear
• User intent matches the tool description

The model sometimes invokes the tool and sometimes ignores it for similar requests.

Actual vs expected behavior:

STT

Expected

I will identify the odd one out

Actual

Old one out

---

silenceDurationMs

Expected

• User turn should end after approximately 6000ms of silence.

Actual

• User turn ends after approximately 2000ms of silence.

---

VAD

Expected

• User should be able to complete a sentence without interruption.
• Brief pauses within a sentence should not trigger turn completion.

Actual

• User speech is occasionally cut off mid-sentence.

---

Tool Calling

Expected

• Consistent tool invocation when user intent clearly matches tool descriptions.

Actual

• Tool calls are inconsistent and appear hit-or-miss.

Any other information you'd like to share?

We have reproduced these behaviors consistently across multiple conversations.

The silence duration issue was specifically validated through timing measurements and conversation logs. The logs indicate that the configured value is received, but the runtime behavior appears to follow the default timeout instead.

Please let us know if these are known limitations/issues of Gemini 3.1 Flash Live Preview or if there are recommended workarounds.

[MTahaRF]

I am also facing the same issue, Please look into this issue as it is breaking our softwares.

[kkorpal]

Hi @AryanKarumuri (and thanks for chiming in, @MTahaRF),

Thanks for the incredibly detailed report! Since 3.1 Flash Live is an experimental preview, I will bring your logs and setup directly to the core Live API engineering team for investigation.

Thanks for your patience and for helping us improve the API!

[AryanKarumuri]

Hey @kkorpal,
Any update on above issues?

[IngLP]

Hi, any update?

[kkorpal]

Hi @AryanKarumuri, following up on this!

To help us investigate the VAD and STT behaviors directly, could you share a minimal code snippet showing your WebSocket setup and audio streaming logic? Having your exact setup will make it much easier to reproduce and track this down. Thanks!