[Feature Request] Allow self-enabling audio in &framegrab mode

[RenzoTsai]

Title: [Feature Request] Allow self-enabling audio in &framegrab mode

Summary

When using VDO.Ninja in &framegrab mode to push a canvas stream, there is currently no apparent way for the publisher to enable their own microphone. The iframe UI does not display a microphone toggle, and URL parameters like &audio or &audiodevice do not seem to initialize an audio track.

However, the core capability to capture audio in this mode clearly exists, as a Director in a room can send a request to the &framegrab user, which successfully prompts them to enable their microphone.

This request is to expose that audio-enabling functionality directly to the publisher, either via a URL parameter or a UI toggle, without requiring a Director's intervention.

Current Behavior

1. A user navigates to a URL with &push=STREAMID&framegrab.
2. The VDO.Ninja iframe loads, ready to receive video frames via postMessage(). This works perfectly for video.
3. The UI presented to this publisher has no microphone icon or audio controls.
4. Adding parameters like &audio=true or &audiodevice=DEVICEID to the URL does not result in audio being captured or in any audio controls appearing in the UI. The resulting stream is video-only.

The Workaround (Demonstrating Capability)

We've observed that audio capture is possible if a Director is involved:

1. The publisher joins a room: &push=STREAMID&framegrab&room=ROOMID.
2. A Director joins the same room.
3. In the Director's UI, they can navigate to the streamer's "Additional Controls" -> "Audio Settings".
4. The Director sends a "request" to the streamer to enable their microphone.
5. This action successfully triggers a microphone permission prompt on the publisher's side, and if accepted, audio is added to the &framegrab stream.

This proves that the &framegrab client is fully capable of handling audio; it just lacks a self-initiation mechanism.

Expected Behavior

A publisher using &framegrab mode should be able to enable their own audio without needing a remote request from a Director.

Ideally, one of the following solutions would be implemented:

• Solution A (URL Parameter): The &framegrab scene should fully respect the &audio and &audiodevice parameters. When the URL includes &audio, it should automatically prompt for microphone access and begin streaming audio alongside the canvas video.
• Solution B (UI Element): A microphone toggle button should be present in the &framegrab iframe UI, allowing the publisher to manually enable, mute, and select their audio device at any time, similar to a standard video scene.

Use Case

Our application is a virtual whiteboard/drawing tool where a presenter draws on an HTML canvas and provides simultaneous audio commentary. The &framegrab feature is perfect for the video component. The presenter, however, is often operating alone and needs a simple way to start streaming their voice along with their drawings, without the complexity of setting up a separate Director's room just to enable their microphone.

Thank you for developing such a powerful and flexible tool!

[steveseguin]

Thank you. I'll see if I can get this working for you. I don't have an ETA, however I've started on it and will try to work on it more as time allows.

[RenzoTsai]

Thank you. That's a great help!