Skip to content

[Fabric] Microsoft.UI.Composition lacks the necessary APIs to bind video content to composition visuals #15631

@bdenney-canon

Description

@bdenney-canon

Problem Description

I am trying to directly render frames to Fabric compositor visual tree natively for custom video frame rendering using zero-copy methods (direct via GPU swapchain).

I have tried several approaches, but so far failed to find a way.

The basic idea is to do the following in native c++ code:

  1. get the Fabric compositor
  2. create a sprite visual
  3. create a surface brush to hold the swapchain
  4. apply the brush to the visual
    5***. get the compositor interop (Fails with E_NOINTERFACE)
  5. Create a composition surface for the swapchain

I am able to do this with Paper architecture, but I don't want to be stuck on old architecture.

Any insights into alternative approaches or future plans/roadmap would be appreciated.

Steps To Reproduce

  1. get the Fabric compositor
  2. create a sprite visual
  3. create a surface brush to hold the swapchain
  4. apply the brush to the visual
    5***. get the compositor interop (Fails with E_NOINTERFACE)

Expected Results

I would expect to be able to render to the composition surface.

CLI version

19.1.2

Environment

npx @react-native-community/cli info
info Fetching system and libraries information...
System:
  OS: Windows 11 10.0.22631
  CPU: (16) x64 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
  Memory: 7.26 GB / 31.72 GB
Binaries:
  Node:
    version: 22.14.0
    path: C:\nvm4w\nodejs\node.EXE
  Yarn:
    version: 1.22.22
    path: C:\Program Files (x86)\Yarn\bin\yarn.CMD
  npm:
    version: 11.8.0
    path: C:\nvm4w\nodejs\npm.CMD
  Watchman: Not Found
SDKs:
  Android SDK: Not Found
  Windows SDK:
    AllowDevelopmentWithoutDevLicense: Enabled
    AllowAllTrustedApps: Enabled
    Versions:
      - 10.0.19041.0
      - 10.0.22621.0
      - 10.0.26100.0
IDEs:
  Android Studio: Not Found
  Visual Studio:
    - 17.14.36705.20 (Visual Studio Professional 2022)
Languages:
  Java:
    version: 11.0.16.1
    path: C:\Program Files\Microsoft\jdk-11.0.16.101-hotspot\bin\javac.EXE
  Ruby: Not Found
npmPackages:
  "@react-native-community/cli":
    installed: 19.1.2
    wanted: ^19.0.0
  react:
    installed: 19.1.0
    wanted: 19.1.0
  react-native:
    installed: 0.81.5
    wanted: 0.81.5
  react-native-windows:
    installed: 0.81.2
    wanted: 0.81.2
npmGlobalPackages:
  "*react-native*": Not Found
Android:
  hermesEnabled: true
  newArchEnabled: true
iOS:
  hermesEnabled: Not found
  newArchEnabled: Not found

info React Native v0.83.2 is now available (your project is running on v0.81.5).
info Changelog: https://github.com/facebook/react-native/releases/tag/v0.83.2
info Diff: https://react-native-community.github.io/upgrade-helper/?from=0.81.5&to=0.83.2
info For more info, check out "https://reactnative.dev/docs/upgrading?os=windows".

Community Modules

"dependencies": {
"@react-native/new-app-screen": "0.81.5",
"protobufjs": "^7.5.4",
"react": "19.1.0",
"react-native": "0.81.5",
"react-native-windows": "0.81.2"
}

Target React Native Architecture

New Architecture (WinAppSDK) Only

Target Platform Version

10.0.22621

Visual Studio Version

Visual Studio 2022

Build Configuration

Debug

Snack, code example, screenshot, or link to a repository

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs: Triage 🔍New issue that needs to be reviewed by the issue management team (label applied by bot)bug

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions