360 Horse example

[dsvilarkovic]

Hello once again,
Can you explain how did you do the horse example from Tanks & Temples dataset to gain such a smooth 360 view of it.
If I assume correctly, you did 180 warping & inpainting from the left side, 180 warping & inpainting from the right side, and then you compiled a video which is reversed 180 left side view and concatenated with 180 right side view?

Besides this, did you also condition your model on a single image, and was the created pointcloud based on single image, or a monocular video?

Do you have some spiral trajectory codebase for this?

Kind regards,
Dusan Svilarkovic

[Chenxi-Song]

Hi Dusan,

Great question—your intuition is basically right.

How we made the 360° view. We separately inferred the left 180° and the right 180° views around the object and concatenated them to obtain a smooth orbit.

Input & geometry. As in the paper, we condition on a single image. We estimate monocular depth with VGGT, then follow a predefined camera path to do the initial view warping + inference. To keep quality high, we use an iterative pass: first produce dense ~30° segments on each side, then re-seed from the tail frame and iterate outward until each side reaches 180°.

We’ll release the warping utilities, the video/inference scripts, and the trajectory helpers (e.g., spiral/orbit paths). The release is taking a bit longer than we hoped because of the code size and some parallel commitments, but we’re actively organizing it and will open-source as soon as it’s ready. Thanks for your patience!

Kind regards,
Chenxi

[dsvilarkovic]

I don't get how do you achieve such a clean 360 view? Is there some additional conditioning on preserving a horse object in a scene? How did you pick a trajectory to work with your video diffusion models and keep the object intact?

Pasting my Gen3C result for the similar 30 by 30 degree attempt.

gen3c_full_360_inpainted.mp4

[Chenxi-Song]

Hi,

Thanks for your interest. To put it simply, the initial input for our horse scene was just a single view. We didn't apply any additional conditioning to preserve the object; the only "condition" was the text prompt itself (e.g., telling the model "it is a horse").

The 360° view is generated iteratively in an "auto-regressive" style. We use the last frame of the previous segment, combined with a warped view from that last frame, as the first frame for generating the next segment.

We have received a lot of curiosity about this panoramic scene generation. We will consider releasing the parameter-setting script for this specific case when we open-source the code. Thanks for following our work.

[dsvilarkovic]

Thanks for replying and explaining your steps. I have no doubt you will have a nice code repo at some point in the future!

Is next segment pointcloud fused to previous pointclouds from previous segment or each segment starts with the new pointcloud every time from single view?

[Chenxi-Song]

Thanks for the encouragement! To answer your question, both approaches work. I usually keep a few views from the previous loops to provide pose reference, but it's also entirely feasible to estimate a new point cloud from just the single last frame.