Long Video [prompts, resolution, other models, interpolation]

[A-2-H]

Great work with the long video generation, it really do great!

I have few questions:

1. Is there a way to incorporate prompt into long video pipline?
   I noticed in the script there is prompt with "". I added args for prompt but I guess there is no difference between adding prompt and not adding it. Is there a way to incorporate it so we can influence generated videos with additional informations like "wind blows hair" or "mist in the background" so it actually have an impact on generated video?

2. Is there an option or configuration to achive higher resolution in square ratio? For example 720x720? If I change the resolution 720x480 to any other it gives an error as the model is trained on 720x480 px...

3. Can we change or implement models to hunyuan or wan? I think those models have more potential with quality and resolution.

4. Should the script interpolate frames form 12 to 30fps? It generates 12 fps videos. Ofcourse I can manually do interpolation to achive 30 fps but I thought in the script there is turned ON option for interpolation of frames or am I missing something?

[gubaoxuan]

Thank you for your attention.

1. The current overall model framework does not support prompt input, and the pipeline with prompts is merely a remnant that has not yet been removed. We will consider adding this feature in the future.
2. The model was trained on a resolution of 720x480 pixels, so changing the resolution to other formats will result in errors. Other resolutions will not be supported in the short term, but we are exploring solutions.
3. We are aware of the potential of models like hunyuan and wan in terms of quality and resolution. We are testing these models and will update our progress.
4. Support for dynamic resolution will be available soon.

[EightiesPower]

I'm going to hijack this thread with my own question - would it be possible to implement a Vid2Vid functionality, like it can be done with LivePortrait? Your model definitely represents an improvement from that model, so if that option was implemented, I could finally retire LP completely. :)

[A-2-H]

Thank you for your attention.

1. The current overall model framework does not support prompt input, and the pipeline with prompts is merely a remnant that has not yet been removed. We will consider adding this feature in the future.
2. The model was trained on a resolution of 720x480 pixels, so changing the resolution to other formats will result in errors. Other resolutions will not be supported in the short term, but we are exploring solutions.
3. We are aware of the potential of models like hunyuan and wan in terms of quality and resolution. We are testing these models and will update our progress.
4. Support for dynamic resolution will be available soon.

Can't wait for future updates, thanks!