Model merging scripts?

[suraj-srinivas]

Hi,

For LORA fine-tuning, are there ways to save only the adapter models and not the full model files? More importantly, what are the easiest ways to perform model merging, given a base model and an adapter?

I am working on a project involving a lot of analysis of fine-tuned models, and saving only the adapters + having on-the-fly model merging functionality would really help.

Thanks for the excellent library!

[pbontrager]

You can look inside of our lora_finetuning recipes to see how we handle checkpointing here. The important function is get_merged_lora_ckpt which will merge your adapter weights with your model weights. The recipe currently saves both the adapters and the merged checkpoints. If you want to save space, you can copy this recipe and modify the save_checkpoint method to only save the adapter weights. Then after training you can merge any adapter you choose with get_merged_lora_ckpt. Let me know if you run into any issues with that.

@ebsmothers I noticed that get_lora_module_names, validate_state_dict_for_lora, get_merged_lora_ckpt, disable_adapter, and validate_missing_and_unexpected_for_lora are not included in the peft init or in the documentation. Could you add those when you get a chance?

[MaxFrax]

@suraj-srinivas @pbontrager I recently encountered the same issue. I'm fine-tuning a LLaMA 3.1 8B model using LoRA and have enabled save_adapter_weights_only as described in the checkpointing guide. Now, I'd like to merge a specific epoch adapter (e.g., adapter_5.pt) with the base model for further testing. Do you have an example or guidance on how to achieve this? I haven’t been able to find a clear solution.