Skip training and run only Multi-LoRA fusion with custom LoRAs

[AgnieszkaPolowczyk]

Hi!

I would like to use your code only for the Multi-LoRA fusion stage from the paper — i.e., to merge multiple LoRA models following Eq.(7).

I already have several LoRA models trained independently (in .safetensors format), so I don’t need to run the LoRA training stage.

Could you please confirm whether this approach is correct?

Should I skip this part completely?

cfr_lora_training(conf.MACE)

And only run the fusion stage?

multi_lora_fusion(conf.MACE)

Thank you very much for providing such a great tool!

[Shilin-LU]

Thank you for your interest!

Yes, if you don’t need to run the LoRA training stage, you can directly call the function multi_lora_fusion to merge multiple LoRA models. You may debug it to check the input format.