ControlNet does not work with Accelerated PyTorch training on Mac :(

[angelkaragiozov]

Hi guys ControlNet does not work with Accelerated PyTorch training on Mac
https://developer.apple.com/metal/pytorch/

NotImplementedError: The operator 'aten::upsample_linear1d.out' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on pytorch/pytorch#77764. As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

P.S.

I found only one solution for Canny edges that works only with SDXL 1.0 -----> https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0

[geroldmeisinger]

HuggingFace implemented a controlnet training script based on accelerate https://huggingface.co/docs/diffusers/training/controlnet
I wrote an article about controlnet training here https://civitai.com/articles/2078 if you want to follow along
I believe it should work on Mac as well