VrRig and display area

[szawel]

Hi, really great job with the Unity plugin.

However, I'm currently encountering an issue when trying to synchronize the camera in Unity with the plugin,
my setup with the Oculus Quest 2 on Win10 RTX 4090
I'm experiencing a "swimming" effect, as if the NeRF does not stick to space, when I move around the stage.
It appears as though the texture from the plugin is being rendered by a stereo camera with settings that differ from those of the Unity rig. Could you advise me on how to approach this issue or suggest ways to minimize or eliminate this problem?
or where to modify the code.

[keli95566]

Hi there! Thank you for the great question, and good catch!

We have noticed this issue and the "swimming" effect is due to the latency between the time when the camera pose is updated and the time the textures are finished rendering.

I have recently figured out a potential solution to solve this issue as I was developing another render plugin for Unity/Cuda/OpenGL.

Basically, we need to use a callback event ( Camrea.onPreCull or Camera.onPreRender) to draw the textures before it appears to user's eyes. In this callback event, we call the plugin event function: GL.IssuePluginEvent(NerfRendererPlugin.GetRenderEventFunc(), DRAW_EVENT); rather than calling this draw event in the update() loop.

I will try to implement this and push the update as soon as possible.

I can will try to integrate the solution in the next next weeks or so.