feat(unit3): Add example of img2img and inpainting implementation #91
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title: feat(unit3): Add example of img2img and inpainting implementation
Body:
This pull request adds example code of a manually implemented denoising loop for img2img and inpainting to the stable_diffusion_introduction.ipynb file in unit3.
Motivation
I was able to understand diffusion models through the tutorial provided in this repository, and I am extremely grateful for that. It was fascinating to see that diffusion models can be applied to tasks like img2img and inpainting. However, it was difficult to grasp the inner workings just by using the pipeline. Therefore, I added a direct implementation to help users understand it more easily.
Detailed Explanation of Implementation
StableDiffusionPipeline
of thediffusers
library, it has been intentionally simplified to focus on the core aspects of the denoising loop for easier understanding by beginners. Unlike the original code that implements the entire loop, non-essential parts have been omitted to concentrate on the noise removal process, minimizing the code's length.runwayml/stable-diffusion-inpainting
is no longer available, the model for inpainting was replaced withstabilityai/stable-diffusion-2-inpainting
.Request
Please review the code for accuracy, efficiency, and readability. In particular, I would appreciate feedback on the implementation of the denoising loop and the masking process for inpainting.