Summary
session.mic.setVoiceActivityDetectionEnabled(true) resolves successfully, but the device continues
to report voiceActivityDetectionEnabled: false on every GLASSES_CONNECTION status event, and
session.mic.onVoiceActivity() never fires.
The call is accepted and appears to do nothing. Because it resolves rather than rejecting, there is
no signal to an app that VAD is unavailable.
Impact
Any client that ends an utterance on VAD going quiet — which the SDK docs suggest, since
onVoiceActivity is documented as true while the user is speaking and false when silent — never
gets its end-of-utterance trigger. In our case every capture ran to our own maximum-utterance
backstop and shipped ~34 seconds of audio:
09:29:17 glasses VAD enabled <- the call resolved
09:29:23 listening (wake)
09:29:54 sending 1086400 bytes (~34.0s, audio/pcm, timeout)
...while the status event ~10 seconds later still said:
"voiceActivityDetectionEnabled": false
Downstream this looked like a model quality problem rather than an audio one — 34 seconds of room
audio transcribes to something, and answers had little to do with the question asked.
Reproduction
await session.mic.setVoiceActivityDetectionEnabled(true) // resolves
session.mic.onVoiceActivity((vad) => console.log("vad", vad.status)) // never fires
session.glasses.onConnection((d) => console.log(JSON.stringify(d)))
// -> "voiceActivityDetectionEnabled": false, on every event
Manifest declares MICROPHONE in both permissions and hardwareRequirements, and
mic.hasPermission is true. Microphone audio itself streams fine via onAudioChunk throughout.
Questions
- Is glasses-side VAD (GX8002) expected to work on Mentra Live, or is
onVoiceActivity only
implemented on other hardware? The docs do not currently gate it by device.
- If it is unsupported, could the call reject rather than resolve? A rejection is actionable; a
silent no-op is not.
docs/bluetooth-sdk/audio states that the Bluetooth SDK "does not apply phone-side Voice Activity
Detection gating to PCM or LC3 events" — is the same true for miniapps? If app-facing audio is
always continuous, it would help to say so next to onVoiceActivity, since the natural reading is
that it can be used for endpointing.
Workaround
We now endpoint on our own audio: per-chunk RMS against a noise floor learned between captures. That
works, and is arguably where it belongs — but it was not obvious that it was necessary, since the
platform advertises the capability and the enable call succeeds.
Environment
@mentra/miniapp 3.1.0-dev.27
- Device: Mentra Live, MentraOS app
staging.20260808.*
- Audio confirmed as PCM 16 kHz, 16-bit mono via
onAudioChunk
Summary
session.mic.setVoiceActivityDetectionEnabled(true)resolves successfully, but the device continuesto report
voiceActivityDetectionEnabled: falseon everyGLASSES_CONNECTIONstatus event, andsession.mic.onVoiceActivity()never fires.The call is accepted and appears to do nothing. Because it resolves rather than rejecting, there is
no signal to an app that VAD is unavailable.
Impact
Any client that ends an utterance on VAD going quiet — which the SDK docs suggest, since
onVoiceActivityis documented astruewhile the user is speaking andfalsewhen silent — nevergets its end-of-utterance trigger. In our case every capture ran to our own maximum-utterance
backstop and shipped ~34 seconds of audio:
...while the status event ~10 seconds later still said:
Downstream this looked like a model quality problem rather than an audio one — 34 seconds of room
audio transcribes to something, and answers had little to do with the question asked.
Reproduction
Manifest declares
MICROPHONEin bothpermissionsandhardwareRequirements, andmic.hasPermissionis true. Microphone audio itself streams fine viaonAudioChunkthroughout.Questions
onVoiceActivityonlyimplemented on other hardware? The docs do not currently gate it by device.
silent no-op is not.
docs/bluetooth-sdk/audiostates that the Bluetooth SDK "does not apply phone-side Voice ActivityDetection gating to PCM or LC3 events" — is the same true for miniapps? If app-facing audio is
always continuous, it would help to say so next to
onVoiceActivity, since the natural reading isthat it can be used for endpointing.
Workaround
We now endpoint on our own audio: per-chunk RMS against a noise floor learned between captures. That
works, and is arguably where it belongs — but it was not obvious that it was necessary, since the
platform advertises the capability and the enable call succeeds.
Environment
@mentra/miniapp3.1.0-dev.27staging.20260808.*onAudioChunk