Native media Equirect layers creation #31033
Conversation
…t loss issue doing it outside the manager. mrdoob#31029
📦 Bundle sizeFull ESM build, minified and gzipped.
🌳 Bundle size after tree-shakingMinimal build including a renderer, camera, empty scene, and dependencies.
|
|
It took a bit to design how to go about it. The only issue is you have to toggle the video mesh layers when going into XR and ending the session. Maybe there is other ways to do it. https://github.com/danrossi/three.js/blob/equirect-layers/examples/webgpu_xr_media_layer.html#L130 live example https://electroteque.org/dev/threejs/examples/webgpu_xr_media_layer.html |
|
I have tried to improve the JSDoc a bit. For a functional review I hope @cabanier can have a look. BTW: Please create a E2E screenshot for the new example |
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
| <head> | ||
| <title>three.js vr - 360 stereo video</title> |
There was a problem hiding this comment.
Please include the new example in files.json. Otherwise it won't appear on the homepage.
|
The requirement to toggle off the video texture with the meshes. I'm not sure how to make better. It causes a crash if the texture is also rendered. |
src/renderers/common/XRManager.js
Outdated
| * @param {boolean} [is180=false] If it's a 180 video. | ||
| * @param {Object} [params={}] Extra params for the layer to add but not needed. | ||
| */ | ||
| createMediaLayer( video, layout = 'mono', transform = {}, is180 = false, params = {} ) { |
There was a problem hiding this comment.
I don't think this code implements emulation for 2D or browser that don't support xr layers. It will also not dynamically add them while in VR.
Please look at the flow of the quad and cylinder layer.
There was a problem hiding this comment.
It does as normal with the meshes added to the scene, just when going to XR the mesh layers need to be disabled, if it doesn't there is a crash in the Oculus browser. I can't get my head around those methods and there is no texture render required.
There is specific logic for the stereo and mono video that shouldn't be in the manager perhaps. There is multiple meshes for the stereo version.
So even if it was created in the manager and added to the scene the meshes need to be disabled.
The native layer only gets setup if layers are supported. Works with the Rift in XR that doesn't support layers it gets rendered in the projection layer.
https://github.com/danrossi/three.js/blob/equirect-layers/examples/webgpu_xr_media_layer.html#L129
https://electroteque.org/dev/threejs/examples/webgpu_xr_media_layer.html
I could automatically disable/enable the mesh layers perhaps. That part I find is little clunky. There is nothing for it to do in the renderLayers method unless it's non layers and not in XR.
If you have another idea it would be good.
There was a problem hiding this comment.
The code that creates the video layer in xr should also the emulation in 2D mode. Right now you create a video texture in your sample code and disable it on session start. That logic needs to go to xrmanager.
This code also doesn't work if you add a media layer while in VR.
There was a problem hiding this comment.
So create the single or multiple mesh in the method. Return and add the geometries before adding to the scene. the issue is while in XR. Is a special render target needed like the others where its then only rendering when not in XR ? so a different render method is needed to the renderLayers method. To prevent that layer disable hack needed. I'm assuming that is the requirement if it actually works like that.
Move equirect layer creation into common layer creation method Fixup example with new media layer usage
|
I've refactored it quite a bit. And followed the structure of modifying the geometry externally to keep that external. I could add that in. It returns a group of meshes which works in 2D, Rift, Quest. There is still the problem of needing to turn off meshes for native layers. Can be done internally or keep it external. I'm not across everything to figure out a better way to render from 2D to native. https://github.com/danrossi/three.js/blob/equirect-layers/examples/webgpu_xr_media_layer.html#L114 https://electroteque.org/dev/threejs/examples/webgpu_xr_media_layer.html mono example https://electroteque.org/dev/threejs/examples/webgpu_xr_media_mono_layer.html |
Related issue: #31029
Description
Adding native media layer creation for WebGPU XR. This fixes a context loss issue doing it outside the manager. There is a potential Oculus browser bug complaining makeXRCompatible isn't set when setting up a media layer still.
Media layers are prepended into the background before the other layers.
Example https://github.com/danrossi/three.js/blob/equirect-layers/examples/webgpu_xr_media_layer.html