Skip to content

miniapp SDK: setVoiceActivityDetectionEnabled(true) resolves but has no effect on Mentra Live; onVoiceActivity never fires #3891

Description

@skthegawd

Summary

session.mic.setVoiceActivityDetectionEnabled(true) resolves successfully, but the device continues
to report voiceActivityDetectionEnabled: false on every GLASSES_CONNECTION status event, and
session.mic.onVoiceActivity() never fires.

The call is accepted and appears to do nothing. Because it resolves rather than rejecting, there is
no signal to an app that VAD is unavailable.

Impact

Any client that ends an utterance on VAD going quiet — which the SDK docs suggest, since
onVoiceActivity is documented as true while the user is speaking and false when silent — never
gets its end-of-utterance trigger. In our case every capture ran to our own maximum-utterance
backstop and shipped ~34 seconds of audio:

09:29:17  glasses VAD enabled            <- the call resolved
09:29:23  listening (wake)
09:29:54  sending 1086400 bytes (~34.0s, audio/pcm, timeout)

...while the status event ~10 seconds later still said:

"voiceActivityDetectionEnabled": false

Downstream this looked like a model quality problem rather than an audio one — 34 seconds of room
audio transcribes to something, and answers had little to do with the question asked.

Reproduction

await session.mic.setVoiceActivityDetectionEnabled(true)   // resolves
session.mic.onVoiceActivity((vad) => console.log("vad", vad.status))  // never fires
session.glasses.onConnection((d) => console.log(JSON.stringify(d)))
// -> "voiceActivityDetectionEnabled": false, on every event

Manifest declares MICROPHONE in both permissions and hardwareRequirements, and
mic.hasPermission is true. Microphone audio itself streams fine via onAudioChunk throughout.

Questions

  1. Is glasses-side VAD (GX8002) expected to work on Mentra Live, or is onVoiceActivity only
    implemented on other hardware? The docs do not currently gate it by device.
  2. If it is unsupported, could the call reject rather than resolve? A rejection is actionable; a
    silent no-op is not.
  3. docs/bluetooth-sdk/audio states that the Bluetooth SDK "does not apply phone-side Voice Activity
    Detection gating to PCM or LC3 events" — is the same true for miniapps? If app-facing audio is
    always continuous, it would help to say so next to onVoiceActivity, since the natural reading is
    that it can be used for endpointing.

Workaround

We now endpoint on our own audio: per-chunk RMS against a noise floor learned between captures. That
works, and is arguably where it belongs — but it was not obvious that it was necessary, since the
platform advertises the capability and the enable call succeeds.

Environment

  • @mentra/miniapp 3.1.0-dev.27
  • Device: Mentra Live, MentraOS app staging.20260808.*
  • Audio confirmed as PCM 16 kHz, 16-bit mono via onAudioChunk

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions