Long Video Training

[nighting0le01]

Hi,

Thanks for releasing this great work!

when will the training code be released?

1. what sink sizes and window size do we use for a latent frame wise autoregressive model?
2. for the long Long Video Training is source anchored DMD an essential part? after naively implementing based on paper, i see some videos collapse to black videos.
3. for the source anchored DMD what is the real_source_guidance_weight? w_src?
4. do we weigh the losses of different segments difrerently? For example do i weigh the clips from further in the rollout more as they might have more error accumulation?

Note that i tried the LHAD from your paper ontop of stage 2 from causal forcing++

[EasonXiao-888]

Thank you for your interest and for further exploration! Here are some answers to your questions:

1. We plan to release the training code in early September under our scheduler. Moreover, we use a sink size of 1 and a window size of 3.
2. In our implementation, SA is an absolutely essential component. It is one of our core methods and plays a critical role in alleviating artifacts and preventing unrecoverable collapse (but we did not meet the black video cases during our training 🤔) during the rollout process. As for the specific implementation details, such as the exact w_source and the weighting strategy across different segments, we will provide the exact configurations in our early September release (Including detailed SA-DMD and LHAD).

Thank you again for trying out our method on top of causal forcing++! If you find our work helpful, citing our paper would be the greatest encouragement for us to continue contributing to the open-source community.
Please stay tuned for the upcoming release!

[nighting0le01]

Hi thanks for your response,
Yes i would be very happy to cite in my work.
do you have ablations with and without SA-DMD? and ablations about how w_src and weighting strategy was selected/ablated ? would be really helpful if you could answer these questions.

[nighting0le01]

also were there any ablations to isolate Chunk-wise Autoregressive Adaptation methods from LHAD? i'm trying to understand if i can directly apply LHAD and SA-DMD on top of my Causal Forcing stage 2 ckpt.