[Config] Do we need to avoid combinatorial number of configs?

[felipemello1]

Currently, for every llama SKU, we have 6 configs:

• LoRA single device
• LoRA distributed
• QLoRA single device
• QLoRA distributed (after FSDP2)
• Full distributed
• Full single device

We do not have DPO/PPO (I guess we probably should), but this could easily add another 6 configs (QLoRA, LoRA, Full) * (distributed, single).

If/when we support DoRA, would we add another 8? (DoRA + QDoRA)*(single, distributed) * (sft, dpo)

Not sure if this needs a solution. Posting it for the discussion. I thought of a few ideas, but i don't think I like them very much

1. Make QLoRA (and other variants) a parameter in LoRA configs?
2. Make model size a parameter? --sku llama8b
3. Only keep single/distributed configs, and make PeFT an optional parameter. We could create a single peft.yaml with just the relevant parameters, shared by all models; 💩
4. Make distributed a parameter? 💩
5. Dont touch it

[gau-nernst]

I think merging QLoRA with LoRA make sense. After all, QLoRA = LoRA + NF4 weight. So the NF4 quant can be an option inside a LoRA recipe. Might help to address #1093 too.