OOM

[Limuyi-col]

My device is an NVIDIA 4090 with 24GB. It will run out of memory (OOM) during the Decode stage. How big VRAM do I need?

[linyq17]

Hi, for inference, it will likely require a GPU with at least 40GB of VRAM to run more reliably.

[Limuyi-col]

Thanks reply！ I've solved this problem. I modified the code in the pipeline_kiwi_edit.py file, optimizing the model loading strategy to offload mode, only loading the corresponding model onto the GPU when needed. I'm just unsure if this will affect the generated results.

[Limuyi-col]

But I encountered another problem. I need to process a long video. Processing all frames at once is impractical, so I have to process it in multiple chunk. However, I find it difficult to control consistency, even when I use overlap. Is there any way to solve this problem?

[linyq17]

Glad to hear that worked!

To check whether the offload-based loading strategy affects result consistency, you could try comparing it with the Hugging Face demo outputs on the same inputs.

For the long-video issue, one practical approach is to subsample frames first instead of processing the entire video densely. Then, after generation, you might use a frame interpolation method to improve temporal fluency and make the final video look smoother.

The consistency problem across chunks is tricky even with overlap, but reducing redundancy through subsampling and adding interpolation afterward may help.