LoRA for Video Generation

[sgonorov]

Context

LoRA adapters work for image generation (see samples/cpp/image_generation/lora_text2image.cpp) but are not yet supported for video generation. The transformer model currently throws an assertion: "Adapters are not currently supported for Video Generation Pipeline" (in src/cpp/src/video_generation/models/ltx_video_transformer_3d_model.cpp).

Several community LoRA adapters for LTX-Video already exist on HuggingFace: https://huggingface.co/models?other=base_model:adapter:Lightricks/LTX-Video

What to do:

• Remove the blocking assertion in ltx_video_transformer_3d_model.cpp and wire up adapter support
• Follow how LoRA is integrated in image generation — see src/cpp/src/lora/adapter.cpp and related helpers in src/cpp/src/lora/
• Ensure the Text2VideoPipeline can accept AdapterConfig through its properties, similar to Text2ImagePipeline
• Add a sample and tests demonstrating LoRA usage with video generation

AI notice — Important! We encourage contributors to use AI tools responsibly when creating Pull Requests. While AI can be a valuable aid, it is essential to ensure that your contributions meet the task requirements, build successfully, include relevant tests, and pass all linters. Submissions that do not meet these standards may be closed without warning to maintain the quality and integrity of the project. Please take the time to understand the changes you are proposing and their impact.

[Eshaan-byte]

Hi @sgonorov can I work on this issue? Thank you!

[sgonorov]

Yeah, feel free to take it. Assigned it.

[goyaladitya05]

Hi @Eshaan-byte, I’ve opened a PR to your fork with fixes to resolve the CI failures. Once you merge those, the PR should pass all the tests. I'm keen to get this in asap, so I can work on #3313, which is blocked until this is approved.