how long to accommodate dream booth fine tuning

[lsjiiia]

thanks

[lucidrains]

@lsjiiia probably by end of first week of september, as I have my hands full with a CUDA project

However, I'm starting to think textual inversion may be the stronger method, and should be added instead of dream booth

What do you think?

[lucidrains]

@lsjiiia What I like about textual inversion is this comment . In other words, you can inject multiple new concepts into your diffusion model, and then perhaps even involve them in the same prompt. I'm not sure if dreambooth can do that

[lucidrains]

Also think textual inversion may have some clue to advance our understanding of continual learning, few-shot learning, as well as memory

[lsjiiia]

I think these two methods are different functions, the dream booth has its own characteristics.
The dream booth is mainly to generate display pictures of the same object with different styles, different attributes, different angles, etc.
The textual inversion may be mainly to learn some repressentation from a small collection of pictures, then use text to guide the generation of other pictures

[lucidrains]

@lsjiiia ohh got it, maybe there's room to build both then!

[deepglugs]

textual inversion sounds really cool

[vanakema]

Dreambooth has far greater ability to create a likeness of a subject. Textual inversion can only produce examples that look like the subject by generating images from what the generator can already produce. It does not learn what a subject looks like, but instead what textual embeddings most closely represent the subject, so it will never look as close to the subject as a Dreambooth implementation

People have said that textual inversion is better at matching styles. Personally I've been trying to get it to generate people, so Dreambooth was a lot better for my usecase

[jgedev]

Any update on this? The Stable Diffusion implementations of Dreambooth have been out for a few months now e.g. https://github.com/XavierXiao/Dreambooth-Stable-Diffusion.

I took a shot at making a PR myself but am struggling to implement the class specific prior preservation loss in Unet. Any ideas?