MediaStream Source/Destination Support for WebRTC Audio Processing

[ibraheem88]

Feature Request

Feature Request: MediaStream Source/Destination Support for WebRTC Audio Processing

🚨 Critical Use Case: Video Calling Applications

This feature is essential for production video calling applications that need to manipulate WebRTC audio streams in real-time. Without it, React Native apps cannot implement advanced audio features that are standard in web applications.

Current Limitation

react-native-audio-api (v0.10.3) does not support creating audio sources from MediaStream objects, which are the standard way WebRTC libraries (like Daily.co, Agora, Twilio) expose audio tracks in React Native.

Missing API Methods

The following Web Audio API methods are not available but are critical for WebRTC audio processing:

• AudioContext.createMediaStreamSource(stream: MediaStream)
• AudioContext.createMediaStreamDestination()
• MediaStreamAudioSourceNode class
• MediaStreamAudioDestinationNode class

Real-World Impact

Our Use Case: Fitness Video Calling Platform

We're building a fitness platform where coaches conduct live video sessions with multiple clients. We need to:

1. Volume Control for One-on-One Calls: When a coach enters a private call with one client, we need to lower (not mute) the volume of other participants to 5% instead of completely unsubscribing from their audio tracks. This is recommended by Daily.co for spatial audio features.

2. Spatial Audio Preparation: Daily.co recommends using Web Audio API gain nodes for spatial audio features, but this is currently impossible in React Native.

3. Dynamic Audio Mixing: Adjust individual participant volumes based on context (active speaker, one-on-one mode, etc.)

Current Workaround (Not Ideal)

We're currently forced to completely unsubscribe from audio tracks, which:

• ❌ Breaks spatial audio features
• ❌ Causes audio dropouts and poor UX
• ❌ Prevents smooth volume transitions
• ❌ Doesn't align with Daily.co's recommendations

Code Example: What We Need

Web Implementation (Currently Working)

On web, we can do this with standardized-audio-context:
pescript
import { AudioContext } from 'standardized-audio-context';

const audioContext = new AudioContext();
const stream = new MediaStream([audioTrack]); // From Daily.co WebRTC

// Create source from WebRTC stream
const source = audioContext.createMediaStreamSource(stream);

// Create gain node for volume control
const gainNode = audioContext.createGain();
gainNode.gain.value = 0.05; // Lower to 5%

// Create destination to output processed audio
const destination = audioContext.createMediaStreamDestination();

// Connect: source -> gain -> destination
source.connect(gainNode);
gainNode.connect(destination);

// Use the processed stream
const processedStream = destination.stream;
audioElement.srcObject = processedStream;

React Native Implementation (Currently Blocked)

import { AudioContext } from 'react-native-audio-api';

const audioContext = new AudioContext();
const stream = new MediaStream([audioTrack]); // From Daily.co WebRTC

// ❌ This doesn't exist yet:
const source = audioContext.createMediaStreamSource(stream); // NOT AVAILABLE
const destination = audioContext.createMediaStreamDestination(); // NOT AVAILABLE

// ✅ These work, but we can't use them without MediaStream support:
const gainNode = audioContext.createGain(); // Works, but no source to connect## Why This Is Urgent

1. Market Demand: Video calling apps are a massive market segment (Zoom, Teams, Daily.co, Agora, Twilio)
2. Feature Parity: Web apps can do this, React Native apps cannot
3. Spatial Audio: This is a prerequisite for the Spatial Audio feature mentioned in your roadmap
4. Production Blocking: Many apps need this for production features (volume control, audio mixing, spatial audio)

Related Features

This feature would enable:

• ✅ Spatial audio (mentioned in your roadmap)
• ✅ Per-participant volume control
• ✅ Audio mixing and effects
• ✅ Noise cancellation integration
• ✅ Echo cancellation
• ✅ Real-time audio processing for video calls

Suggested Implementation Priority

High Priority - This unlocks a major use case category (video calling apps) and is a prerequisite for spatial audio features.

Additional Context

• Library: Daily.co React Native SDK
• WebRTC: Using @daily-co/react-native-webrtc
• Current Version: react-native-audio-api@0.10.3
• Platforms: iOS and Android both need this

References

• Daily.co Spatial Audio Docs: Recommends Web Audio API for spatial audio
• Web Audio API Spec: https://www.w3.org/TR/webaudio/#mediastreamaudiosourcenode
• Related Issue: This blocks spatial audio implementation mentioned in roadmap

---

Impact: Without this feature, React Native video calling apps cannot implement professional-grade audio features that are standard in web applications. This significantly limits the library's adoption in one of the fastest-growing app categories.

Thank you for considering this feature request. This would be a game-changer for video calling applications on React Native! 🚀

[maciejmakowski2003]

already present in our road map. it should be added with v0.13 (maybe v0.12).

[ibraheem88]

Thank you for the update!