FSDP2 and peft

[psinger]

Hey, sorry if this is the wrong place. Feel free to move it to discussion.

I am trying to get peft working with fsdp2 and am wondering if someone else attempted that already?

The issue is that Im always getting errors along the lines of:
RuntimeError: aten.mm.default: got mixed torch.Tensor and DTensor, need to convert all torch.Tensor to DTensor before calling distributed operators!

Happy for any pointers.

[BenjaminBossan]

Could you please provide environment info (package versions, hardware, etc), a reproducer and the full error message? In case you're using accelerate (maybe indirectly via transformers Trainer), please take note that FSDP2 is not supported yet.

[psinger]

No, Im not using accelerate. But Im following torchtriton:
https://github.com/pytorch/torchtune/blob/main/recipes/lora_dpo_distributed.py

This is more an explorative question if someone has successfully run fsdp2 with peft explicitly? I am not finding any info out there on it.

I think it mostly boils down which layers etc youre sharding at what point.

[BenjaminBossan]

But Im following torchtriton:
https://github.com/pytorch/torchtune/blob/main/recipes/lora_dpo_distributed.py

Unless you've rewritten that script, note that it uses torchtune, not PEFT.

This is more an explorative question if someone has successfully run fsdp2 with peft explicitly?

Personally, I haven't tried it. I'll probably do that once FSDP2 is supported by accelerate.

[psinger]

That's why Im asking here, I tried using peft with pretty much their script. It seems to work for them with their lora implementations.

[BenjaminBossan]

I see. Again, without a reproducer, it's going to be hard to help investigate the issue. Also, providing the full error message could be helpful. Otherwise, I'd wait for the accelerate support for FSDP2, we're probably going to add FSDP2 examples (or update existing FSDP examples) once that is ready.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.