issues when training

[Urheen]

Hi when caluclating

grad, = autograd.grad(
outputs=(fake_img * noise).sum(), inputs=latents, create_graph=True
)

I got this error:

RuntimeError: derivative for aten::_scaled_dot_product_flash_attention_backward is not implemented

Have you faced any errors for this?

[JiayiGuo821]

autograd requires disabling flash attention.