Replace deprecated stabilityai/stable-diffusion-2-* models with ones maintained by sd2-community.

Add disclaimers about unofficial mirror used instead.

Signed-off-by: Artur Kloniecki <arturx.kloniecki@intel.com>

Author: AKloniecki

docs/source/tutorials/stable_diffusion.mdx

@@ -67,13 +67,15 @@ Check out the [example](/examples/stable-diffusion) provided in the official Git
 
 ## Stable Diffusion 2
 
+DISCLAIMER: Stable Diffusion 2 models family has been discontinued and withdrawn by Stability AI. The following instruction uses mirrored models maintained by sd2-community, which is not affiliated in any way with Stability AI. Follow these instructions at your own risk.
+
 [Stable Diffusion 2](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_2) can be used with the exact same classes.
 Here is an example:
 
 ```python
 from optimum.habana.diffusers import GaudiDDIMScheduler, GaudiStableDiffusionPipeline
 
-model_name = "stabilityai/stable-diffusion-2-1"
+model_name = "sd2-community/stable-diffusion-2-1"
 
 scheduler = GaudiDDIMScheduler.from_pretrained(model_name, subfolder="scheduler")
 
@@ -96,10 +98,10 @@ outputs = pipeline(
 
 <Tip>
 
-There are two different checkpoints for Stable Diffusion 2:
+There are two different checkpoints for Stable Diffusion 2 ():
 
-- use [stabilityai/stable-diffusion-2-1](https://huggingface.co/stabilityai/stable-diffusion-2-1) for generating 768x768 images
-- use [stabilityai/stable-diffusion-2-1-base](https://huggingface.co/stabilityai/stable-diffusion-2-1-base) for generating 512x512 images
+- use [sd2-community/stable-diffusion-2-1](https://huggingface.co/sd2-community/stable-diffusion-2-1) for generating 768x768 images
+- use [sd2-community/stable-diffusion-2-1-base](https://huggingface.co/sd2-community/stable-diffusion-2-1-base) for generating 512x512 images
 
 </Tip>
 

examples/stable-diffusion/depth_to_image_generation.py

@@ -52,7 +52,8 @@ def main():
 
     parser.add_argument(
         "--model_name_or_path",
-        default="stabilityai/stable-diffusion-2-depth",
+        # Stability AI has removed stable-diffusion-2 models. This uses unofficial mirror by sd2-community
+        default="sd2-community/stable-diffusion-2-depth",
         type=str,
         help="Path to pre-trained model",
     )

examples/stable-diffusion/training/README.md

@@ -91,11 +91,15 @@ To download the example conditioning images locally, run:
 python download_train_datasets.py
 ```
 
+> [!DISCLAIMER]
+> Stable Diffusion 2 models family has been discontinued and withdrawn by Stability AI. The following instruction uses mirrored models maintained by sd2-community, which is not affiliated in any way with Stability AI.
+> Follow these instructions at your own risk.
+
 Then proceed to training with command:
 
 ```bash
 PT_HPU_LAZY_MODE=1 python train_controlnet.py \
-   --pretrained_model_name_or_path=stabilityai/stable-diffusion-2-1 \
+   --pretrained_model_name_or_path=sd2-community/stable-diffusion-2-1 \
    --output_dir=/tmp/stable_diffusion2_1 \
    --dataset_name=fusing/fill50k \
    --resolution=512 \
@@ -116,11 +120,15 @@ with `python ../../gaudi_spawn.py --world_size <num-HPUs> train_controlnet.py`.
 
 ### Inference
 
+> [!DISCLAIMER]
+> Stable Diffusion 2 models family has been discontinued and withdrawn by Stability AI. The following instruction uses mirrored models maintained by sd2-community, which is not affiliated in any way with Stability AI.
+> Follow these instructions at your own risk.
+
 After training completes, you can use `text_to_image_generation.py` sample to run inference with the fine-tuned ControlNet model:
 
 ```bash
 PT_HPU_LAZY_MODE=1 python ../text_to_image_generation.py \
-    --model_name_or_path stabilityai/stable-diffusion-2-1 \
+    --model_name_or_path sd2-community/stable-diffusion-2-1 \
     --controlnet_model_name_or_path /tmp/stable_diffusion2_1 \
     --prompts "pale golden rod circle with old lace background" \
     --control_image "./cnet/conditioning_image_1.png" \
@@ -224,9 +232,13 @@ python download_train_datasets.py
 
 To launch the multi-card Stable Diffusion training, use:
 
+> [!DISCLAIMER]
+> Stable Diffusion 2 models family has been discontinued and withdrawn by Stability AI. The following instruction uses mirrored models maintained by sd2-community, which is not affiliated in any way with Stability AI.
+> Follow these instructions at your own risk.
+
 ```bash
 PT_HPU_LAZY_MODE=1 python ../../gaudi_spawn.py --world_size 8 --use_mpi train_dreambooth.py \
-    --pretrained_model_name_or_path="stabilityai/stable-diffusion-2-1"  \
+    --pretrained_model_name_or_path="sd2-community/stable-diffusion-2-1"  \
     --instance_data_dir="dog" \
     --output_dir="dog_sd" \
     --class_data_dir="path-to-class-images" \
@@ -263,9 +275,13 @@ UNet or text encoder.
 
 To run the multi-card training, use:
 
+> [!DISCLAIMER]
+> Stable Diffusion 2 models family has been discontinued and withdrawn by Stability AI. The following instruction uses mirrored models maintained by sd2-community, which is not affiliated in any way with Stability AI.
+> Follow these instructions at your own risk.
+
 ```bash
 PT_HPU_LAZY_MODE=1 python ../../gaudi_spawn.py --world_size 8 --use_mpi train_dreambooth.py \
-    --pretrained_model_name_or_path="stabilityai/stable-diffusion-2-1"  \
+    --pretrained_model_name_or_path="sd2-community/stable-diffusion-2-1"  \
     --instance_data_dir="dog" \
     --output_dir="dog_sd" \
     --class_data_dir="path-to-class-images" \
@@ -310,7 +326,7 @@ After training completes, you can use `text_to_image_generation.py` sample for i
 
 ```bash
 PT_HPU_LAZY_MODE=1 python ../text_to_image_generation.py \
-    --model_name_or_path stabilityai/stable-diffusion-2-1  \
+    --model_name_or_path sd2-community/stable-diffusion-2-1  \
     --unet_adapter_name_or_path dog_sd/unet \
     --prompts "a sks dog" \
     --num_images_per_prompt 5 \

optimum/habana/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_depth2img.py

@@ -368,7 +368,7 @@ def __call__(
         >>> from diffusers import StableDiffusionDepth2ImgPipeline
 
         >>> pipe = StableDiffusionDepth2ImgPipeline.from_pretrained(
-        ...     "stabilityai/stable-diffusion-2-depth",
+        ...     "sd2-community/stable-diffusion-2-depth",
         ...     torch_dtype=torch.float16,
         ... )
         >>> pipe.to("cuda")

optimum/habana/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py

@@ -339,7 +339,7 @@ def __call__(
         >>> mask_image = download_image(mask_url).resize((512, 512))
 
         >>> pipe = StableDiffusionInpaintPipeline.from_pretrained(
-        ...     "stabilityai/stable-diffusion-2-inpainting", torch_dtype=torch.float16
+        ...     "sd2-community/stable-diffusion-2-inpainting", torch_dtype=torch.float16
         ... )
         >>> pipe = pipe.to("cuda")
 

tests/test_diffusers.py

@@ -793,7 +793,8 @@ def test_no_throughput_regression_autocast(self):
         ]
         num_images_per_prompt = 28
         batch_size = 7
-        model_name = "stabilityai/stable-diffusion-2-1"
+        # Stability AI has removed stable-diffusion-2 models. This uses unofficial mirror by sd2-community
+        model_name = "sd2-community/stable-diffusion-2-1"
         scheduler = GaudiDDIMScheduler.from_pretrained(model_name, subfolder="scheduler")
         pipeline = GaudiStableDiffusionPipeline.from_pretrained(
             model_name,
@@ -2635,7 +2636,8 @@ def test_depth2img_pipeline_hpu_graphs(self):
     @legacy
     def test_depth2img_pipeline(self):
         gaudi_config = GaudiConfig(use_torch_autocast=True)
-        model_name = "stabilityai/stable-diffusion-2-depth"
+        # Stability AI has removed stable-diffusion-2 models. This uses unofficial mirror by sd2-community
+        model_name = "sd2-community/stable-diffusion-2-depth"
         scheduler = GaudiDDIMScheduler.from_pretrained(model_name, subfolder="scheduler")
 
         pipe = GaudiStableDiffusionDepth2ImgPipeline.from_pretrained(
@@ -5687,7 +5689,8 @@ def test_stable_diffusion_inpaint_no_throughput_regression(self):
         prompts = [
             "concept art digital painting of an elven castle, inspired by lord of the rings, highly detailed, 8k",
         ]
-        model_name = "stabilityai/stable-diffusion-2-inpainting"
+        # Stability AI has removed stable-diffusion-2 models. This uses unofficial mirror by sd2-community
+        model_name = "sd2-community/stable-diffusion-2-inpainting"
         num_images_per_prompt = 12
         batch_size = 4
         pipeline = GaudiStableDiffusionInpaintPipeline.from_pretrained(