RunPod Fix For DreamBooth & xFormers - How To Use Automatic1111 Web UI Stable Diffusion on RunPod

[FurkanGozukara]

RunPod Fix For DreamBooth & xFormers - How To Use Automatic1111 Web UI Stable Diffusion on RunPod

Full tutorial: https://www.youtube.com/watch?v=zA4LksIVas8

This video outdated. New tutorial : https://youtu.be/c_S2kFAefTQ

RunPod: https://bit.ly/RunPodIO. Discord : https://bit.ly/SECoursesDiscord. How to install latest version of Automatic1111 Web UI and #DreamBooth on #RunPod. If I have been of assistance to you and you would like to show your support for my work, please consider becoming a patron on 🥰 https://www.patreon.com/SECourses

Playlist of #StableDiffusion Tutorials, Automatic1111 and Google Colab Guides, DreamBooth, Textual Inversion / Embedding, LoRA, AI Upscaling, Pix2Pix, Img2Img:

https://www.youtube.com/playlist?list=PL_pbwdIyffsmclLl0O144nQRnezKlNdx3

Commands having GitHub file : https://github.com/FurkanGozukara/Stable-Diffusion/blob/main/Tutorials/How-To-Install-DreamBooth-Extension-On-RunPod.md

DreamBooth extension: https://github.com/d8ahazard/sd_dreambooth_extension

Longer RunPod Tutorial : https://youtu.be/QN1vdGhjcRc

00:00:00 How to fix SD Web UI on RunPod and use DreamBooth extension tutorial

00:00:14 Which RunPod you should pick for DreamBooth training

00:00:20 Which RunPod template you should pick for Stable Diffusion Automatic1111 Web UI and DreamBooth

00:01:00 How to connect your RunPod

00:01:03 How to update your Stable Diffusion Automatic1111 web UI

00:01:17 How you should modify relauncher.py file and why

00:01:31 How to install DreamBooth extension manually

00:02:07 How to fix requirements version conflicting error

00:03:06 How to install correct CUDA and Torch version on RunPod

00:03:35 How to install correct xFormers version on RunPod

00:04:12 How to install requirements of DreamBooth extension manually

00:04:40 How to download best model for DreamBooth training

00:04:58 How to download best VAE file

00:05:20 How to restart your RunPod

00:05:38 How to terminate initially started SD Web UI and start your own SD Web UI

00:06:02 How to start WEB UI instance in RunPod

00:06:18 How to connect your WEB UI instance with the fastest way

00:06:54 How to do a DreamBooth training on RunPod

00:07:15 How to set default VAE file - best one

00:07:45 Best training settings for DreamBooth training

00:09:42 Pre-generated 6000+ photo of man classification images

00:10:17 How to fix out of memory error caused by a memory leak

Emergence of Innovative AI Technologies: RunPod, Stable Diffusion, Xformers, DreamBooth, Textual Inversion, DALL-E, Midjourney, and PyTorch

Introduction

Artificial intelligence (AI) has rapidly evolved over the last few years, giving rise to a plethora of novel technologies and applications. In this article, we will delve into eight groundbreaking AI advancements: RunPod, Stable Diffusion, Xformers, DreamBooth, Textual Inversion, DALL-E, Midjourney, and PyTorch. Each of these technologies showcases the versatility and power of AI, shaping the future of various industries.

RunPod

RunPod is an AI-powered platform that optimizes the execution of machine learning (ML) models. It streamlines the deployment and management of ML workflows, reducing complexity and accelerating the development process. RunPod achieves this by automatically selecting the most suitable computational resources, minimizing latency, and ensuring optimal performance. Consequently, RunPod enables organizations to implement efficient ML systems and focus on their core business objectives.

Stable Diffusion

Stable Diffusion is a novel technique for training generative models in AI. It enhances the stability and convergence of training processes, yielding more accurate and realistic output. This approach is particularly beneficial for image and video synthesis tasks, where traditional methods may struggle with high-dimensional data. Stable Diffusion improves upon the existing diffusion models by introducing new optimization techniques, resulting in faster training times and improved sample quality. As a result, Stable Diffusion has the potential to revolutionize computer vision and graphics applications.

Xformers

Xformers is a new family of transformer-based AI models designed to handle large-scale data more efficiently. These models address the limitations of traditional transformers, such as high computational complexity and memory requirements. Xformers leverage novel techniques like sparse attention, adaptive computation, and efficient matrix operations to reduce resource consumption without sacrificing performance. This enables the deployment of advanced AI models on resource-constrained devices, opening the door for real-time applications in areas such as natural language processing, computer vision, and speech recognition.

DreamBooth

DreamBooth is an AI-driven virtual reality (VR) platform that generates immersive and highly realistic environments. It combines the power of generative models, like GANs (Generative Adversarial Networks), with advanced VR technologies to create customizable, interactive experiences. Users can modify the virtual environment in real-time, thanks to the platform's ability to generate new content on the fly.

Video Transcription

• 00:00:00 Greetings everyone, there have been so many  Automatic1111 Web UI and DreamBooth Extension  

• 00:00:04 updates. Therefore, in this video, I will show  you how you can use the latest Automatic1111  

• 00:00:09 Web UI and DreamBooth Extension with the latest  xFormers on a RunPod. For DreamBooth training,  

• 00:00:16 I am suggesting you to get at least 20GB VRAM  and 30GB RAM memory. So click Deploy. From here,  

• 00:00:22 select your template as Stable Diffusion 1.5  version. Make the temporary disk as 10GB because  

• 00:00:29 we are going to do some installation. Make the  persistent volume as you wish and deploy. This  

• 00:00:34 will be a short video. Therefore, if you want to  learn more about RunPod and how to use it with  

• 00:00:40 Stable Diffusion and Automatic1111 Web UI, I have  an excellent tutorial. It is very long and it is  

• 00:00:46 very comprehensive. Watching this tutorial will  help you significantly. Okay, after creating our  

• 00:00:51 Pod, let's go to the My Pods. In here, it will  be get initialized. Once the initialization is  

• 00:00:57 completed, we will see connect button here. Okay,  connect button appeared. Let's connect. Connect  

• 00:01:02 to JupyterLab. Okay, let's begin with updating our  Stable Diffusion Web UI. Enter inside your Stable  

• 00:01:08 Diffusion Web UI folder. Click Terminal. Your  terminal will start in this folder. Then type git  

• 00:01:14 pull. This will update it to latest version. Then  we are going to update relauncher.py. Double click  

• 00:01:20 it. It will be opened like this. It is constantly  trying to relaunch. Therefore, we will change the  

• 00:01:25 behavior. (n < 1). Click save with ctrl s. It  is saved. Then we will install the DreamBooth  

• 00:01:32 extension manually. To do that, copy the URL of  the extension from this gist file. I will put this  

• 00:01:40 gist file link in the description. This is really  important one. Copy. Then inside this extensions  

• 00:01:46 folder, click plus icon. Click Terminal. The  terminal will be opened inside this folder.  

• 00:01:52 Therefore, just type git clone. Paste the URL. The  extension has been cloned here. The very important  

• 00:01:59 thing is that in the Unix environment, there is  a problem. The problem is that the requirements  

• 00:02:07 versions. There is a requirements conflicting  happening. Therefore, enter inside your DreamBooth  

• 00:02:13 extension folder. You will see requirements.txt  file. Open it. When you double click it, it will  

• 00:02:18 be opened. Then go back to your Stable Diffusion  Web UI. In here, you will see requirements  

• 00:02:23 versions.txt file. So you need to check each one  of them and fix the conflictions. For example,  

• 00:02:30 the Accelerate version of the DreamBooth extension  is 0.17.1. And the Accelerate version of the  

• 00:02:37 Stable Diffusion Web UI is 0.12. Therefore,  we will override it with the version of the  

• 00:02:43 DreamBooth extension. There is also transformers.  So I will copy paste the transformers version as  

• 00:02:48 well like this. And there is also one more. This  is GitPython. I am also copy pasting it. Why we  

• 00:02:54 are doing this? Because when you are launching  your Web UI, it will override back to the lower  

• 00:02:59 version and it will cause the DreamBooth  extension not work. Once you have done this,  

• 00:03:04 we will make two installation. Which is the  correct Torch and CUDA version and the xFormers  

• 00:03:11 version. Both of them are posted on our Gist file.  The first one we are going to install is this one.  

• 00:03:18 Torch version. So click plus icon, open terminal.  You can open this terminal inside your workspace.  

• 00:03:24 It will work because by default it is using the  venv folder, virtual environment folder of the  

• 00:03:30 Stable Diffusion Web UI installation. And you  see it is already installed by default. Then the  

• 00:03:35 second command we are going to run is installing  this specific xFormers. For finding this,  

• 00:03:42 I have spent a lot of time. Actually, I have spent  over six hours to figure out the correctly working  

• 00:03:47 xFormers version because they have removed the  other xFormers wheels from the repository. So  

• 00:03:54 first we will uninstall the xFormers version  with pip uninstall xFormers. It will uninstall  

• 00:04:00 the xFormers. It will ask you to continue.  Click proceed. Yes, and it is uninstalled.  

• 00:04:05 Then copy paste the command I posted on the Gist  file. It will install the latest xFormers. OK,  

• 00:04:13 there is one more thing left, which is installing  the requirements of the DreamBooth extension. So  

• 00:04:18 go inside your extensions, inside your DreamBooth  extension folder. Click plus. Open a new terminal.  

• 00:04:25 You see currently my terminal is inside extensions  folder. Type pip install dash r requirements txt  

• 00:04:33 as I am showing here. Copy it and paste it. And  it will install the requirements of the DreamBooth  

• 00:04:40 extension. Then meanwhile, let's also download the  best model for training. Go inside your models.  

• 00:04:46 Inside Stable Diffusion, open a new terminal.  Then copy paste this line, which will download  

• 00:04:53 the 1.5 pruned ckpt file. Copy and paste it. It  will download it. Let's also download the best VAE  

• 00:05:00 file. To VAE file, go to the models. Go to the VAE  folder. Copy this link. Then inside VAE folder,  

• 00:05:07 open a new terminal. Paste it and it will download  it here. OK, so the download of the VAE file is  

• 00:05:14 completed. The download of the model is completed.  And the installation of the DreamBooth extension  

• 00:05:20 requirements has been completed. Now we will  restart our pod. To do that, go to the restart pod  

• 00:05:27 like this. After restart has been completed, click  connect. Connect to JupyterLab. This may take a  

• 00:05:33 while. So when you restart or start your RunPod,  it will start an instance of web UI. We are going  

• 00:05:39 to terminate that web UI with this command, fuser  -k 3000/tcp. So to do that, go to inside your  

• 00:05:47 Stable Diffusion web UI. Click plus icon. Open  a new terminal. Type it. It is killed now. Once  

• 00:05:53 your existing web UI is terminated, you will see  that the GPU memory used is 0%. Now we will start  

• 00:06:01 the web UI ourselves. To do that, you need to type  python relauncher.py. This has to be called inside  

• 00:06:07 Stable Diffusion web UI folder. It will restart  your web UI. You will get some warnings like this,  

• 00:06:13 but you can ignore them. Web UI has been  started. There is no errors. Then for connecting,  

• 00:06:20 I am going to use connect and 3000 HTTP port. This  is working a little bit faster. And now everything  

• 00:06:27 is loaded. Let's try 1.5 pruned ckpt file. Also,  currently we are using the latest version of  

• 00:06:35 Automatic1111 web UI. The latest version of  the xFormers. Unfortunately, Torch version 2  

• 00:06:41 is still not working on RunPod. Therefore, we are  using Torch 1.13 and CUDA 11.7. But this is also  

• 00:06:48 working very fast. Then make a test. Car. So image  has been generated. Let's go to the DreamBooth.  

• 00:06:54 Let's compose a model. Test. Source checkpoint.  1.5 pruned ckpt, unfreeze model. Create model.  

• 00:07:01 With this way, you are able to see the CMD, the  command line interface of the web UI. If you  

• 00:07:07 don't terminate it yourself, you won't be able to  see that in this screen. So this approach is best  

• 00:07:13 way for RunPod. By the way, let's also set the  VAE file. Go to the settings. In here, you will  

• 00:07:19 see Stable Diffusion. SD VAE you see. Make it this  downloaded VAE file and click apply settings. It  

• 00:07:27 is changed. Let's go back to our DreamBooth. Okay,  the model has been generated and it is selected.  

• 00:07:32 So I will make a quick demonstration for you.  Let's make a new folder, training images. Let  

• 00:07:38 me upload my files there. I will just drag and  drop them. Okay, the files are uploaded. Let's  

• 00:07:44 make the best settings. First click performance  wizard, then concept training wizard. So the best  

• 00:07:49 settings I have found are like this. I'm not  changing anything. By the way, these settings  

• 00:07:54 are set for 20 GB. So it may change based on your  VRAM. So this is 1e-7. In here, use EMA. I will  

• 00:08:03 use Lion. bf16. I am not going to use xFormers,  but you can use that as well. Cache latents,  

• 00:08:08 train UNET. This is really important. Step ratio  text encoders, 75%. And weight decay will be 0.1%  

• 00:08:17 because we are using Lion. And when you are  using Lion, this is the learning rate. Okay,  

• 00:08:22 for quickly starting the training, let's copy the  path of the train images. Paste it there. Don't  

• 00:08:29 forget the backslash. Copy and paste it like this.  And let's make it class. ohwx man is our instance  

• 00:08:36 token. Photo of man is our class token. Normally  you would need 50. But for quick test, I will  

• 00:08:43 make it 1. So we will see. It will generate the  classification images. But using text to image tab  

• 00:08:50 is better for generating classification images. I  am explaining that in my other videos. So then go  

• 00:08:56 to the saving tab. Generate a ckpt. We can also  make this like 200 and save every 10 epochs. Then  

• 00:09:02 you can compare epochs. So I will save previews  every one epoch to demonstrate to you. But I am  

• 00:09:10 usually making this zero then I am using xyz plot  checkpoint comparison. Save settings and click  

• 00:09:16 load settings to not get any error. Then train.  Okay, first it will generate the classification  

• 00:09:22 images. Since we did set as one, it will  generate 17. It is generating them right now. Oh,  

• 00:09:28 by the way, we have forgotten the sample image  prompt. Therefore, it won't be able to generate  

• 00:09:33 correct images. We should have typed here photo of  ohwx man. So I will just cancel and then restart.  

• 00:09:42 I also have a sample data set which has 6000  images. And I posted that on our Patreon page.  

• 00:09:49 You will see our Patreon page both in the comments  and the description of this video. You can click  

• 00:09:55 the Patreon. Then on our Patreon, you will see  this post. Okay, you see the training has started.  

• 00:10:01 Now I will just cancel. Okay, then save settings,  load settings. You see there is some random image  

• 00:10:07 because we have forgotten to set sample image  prompt. And then train. This time it won't  

• 00:10:12 generate any more classification images because we  already have them generated. Okay, you see I have  

• 00:10:19 an error this time out of memory error. I think  this is because of memory leakage. So therefore,  

• 00:10:25 I will restart pod and start again. Okay, after  restart, it started training and it is using  

• 00:10:31 18.1 gigabytes VRAM currently. We are not using  gradient checkpointing. When you enable it, it  

• 00:10:38 will also reduce VRAM usage. And the first sample  is also generated very well. Okay, after just 20  

• 00:10:45 epochs, I am able to get very resembling images.  And as the training continues, it will get better.  

• 00:10:51 However, I have noticed that I used the rare token  incorrectly. So be careful with that. Whatever the  

• 00:10:57 rare token you teach, you need to use that. Both  in the sample images and both in the text to image  

• 00:11:04 tab. So DreamBooth extension currently working  very well and very fast. And the stable diffusion  

• 00:11:10 and the Web UI working very well and very fast  on the RunPod with this methodology, I just have  

• 00:11:15 shown in this video. If you have enjoyed this  video, please like, subscribe, join our channel.  

• 00:11:21 If you join us and support us, I would appreciate  that very much. Please also become a patron of us  

• 00:11:27 and support me. I would appreciate that very much.  And it is also helping me tremendously. I have so  

• 00:11:33 many good tutorials about Stable Diffusion. You  can watch all of them and learn many stuffs and  

• 00:11:39 they will help you tremendously. Hopefully see  you in another awesome video. And I am planning  

• 00:11:45 a very advanced tutorial for how to install  and use Automatic1111 Web UI on a PC with  

• 00:11:52 the latest settings, latest configuration,  latest libraries. Hopefully see you later.