What I'm trying to do
Embed Daily-hosted real-time video (specifically Tavus rooms, which run on Daily) inside a visionOS RealityKit immersive space. The goal is to bind a remote participant's video track to a RealityKit LowLevelTexture so an avatar appears as a textured plane in 3D space, with hand-anchored placement and spatial HRTF audio.
What's blocking me
Package.swift at v0.37.0 declares platforms: [.iOS(.v13)] only — no visionOS platform support.
Even forcing the iOS framework into a visionOS app via "Designed for iPad" compatibility, the published API surface is UIKit-bound (CallClient.makeView() returns a UIView); there is no public path to grab the remote video track as CVPixelBuffer / CMSampleBuffer / AVCaptureVideoDataOutput-style stream that we could bridge into a LowLevelTexture for RealityKit rendering.
Today the WebRTC stack works fine if you embed Daily inside a WKWebView in a flat WindowGroup, but you lose every reason to be on visionOS — no spatial anchoring, no walk-around, no parallax, no HRTF.
What I'm asking
- Add visionOS to the supported platforms in
Package.swift. Even with no spatial-specific APIs, having the framework declared compatible would unblock projects that just want to consume Daily rooms in a windowed visionOS app.
- Expose a raw video-track callback / publisher that delivers
CMSampleBuffer or CVPixelBuffer for remote participants, so consumers can route the stream into custom rendering pipelines (RealityKit LowLevelTexture.update(), Metal, etc.).
- Expose a raw audio-track callback that delivers PCM frames so consumers can route audio through
PHASE / AVAudioEngine for HRTF-spatial placement at a 3D anchor.
Even (1) alone unblocks a huge class of "embedded Daily on visionOS" use cases. (2) and (3) are the unlock for native spatial avatars.
Context
We're building a visionOS native app where a Tavus avatar should live at a user-placed anchor in their physical office — not in a window, but in the room. Currently shipping the WebView panel as a stopgap inside a RealityView attachment; would prefer the real path.
Happy to help test or contribute a sample integration if useful.
What I'm trying to do
Embed Daily-hosted real-time video (specifically Tavus rooms, which run on Daily) inside a visionOS RealityKit immersive space. The goal is to bind a remote participant's video track to a RealityKit
LowLevelTextureso an avatar appears as a textured plane in 3D space, with hand-anchored placement and spatial HRTF audio.What's blocking me
Package.swiftat v0.37.0 declaresplatforms: [.iOS(.v13)]only — no visionOS platform support.Even forcing the iOS framework into a visionOS app via "Designed for iPad" compatibility, the published API surface is UIKit-bound (
CallClient.makeView()returns aUIView); there is no public path to grab the remote video track asCVPixelBuffer/CMSampleBuffer/AVCaptureVideoDataOutput-style stream that we could bridge into aLowLevelTexturefor RealityKit rendering.Today the WebRTC stack works fine if you embed Daily inside a
WKWebViewin a flatWindowGroup, but you lose every reason to be on visionOS — no spatial anchoring, no walk-around, no parallax, no HRTF.What I'm asking
Package.swift. Even with no spatial-specific APIs, having the framework declared compatible would unblock projects that just want to consume Daily rooms in a windowed visionOS app.CMSampleBufferorCVPixelBufferfor remote participants, so consumers can route the stream into custom rendering pipelines (RealityKitLowLevelTexture.update(), Metal, etc.).PHASE/AVAudioEnginefor HRTF-spatial placement at a 3D anchor.Even (1) alone unblocks a huge class of "embedded Daily on visionOS" use cases. (2) and (3) are the unlock for native spatial avatars.
Context
We're building a visionOS native app where a Tavus avatar should live at a user-placed anchor in their physical office — not in a window, but in the room. Currently shipping the WebView panel as a stopgap inside a RealityView attachment; would prefer the real path.
Happy to help test or contribute a sample integration if useful.