Trained CKPT - always same results

[akilmaf]

Hi there,

First of all thank you for your contribution. I learned a lot and still learning. My problem is that after preparing everything and training on the pictures (23 photos, close shot) I was finally able to run the ckpt and generate new visuals. But every time I click generate, the result is always almost the same photo I uploaded while training. Same face & cloth. It acts like copying my photo and pasting after generating.

I created 3 ckpt files (spent 2 days...) so far with different colab tutorials. None of them worked visually. Where do I make mistake?

Thank you in advance.

[MasterDenis]

Hi, did you use regularization images? ("contains faces"),
Also How much was the number in your text encoder settings?

[akilmaf]

Hi,

• For regularization images, I could not find a place to upload them. Maybe the interface has been updated? Anyways I downloaded regularization images samples from this link, found them in a tutorial video, https://github.com/aitrepreneur/REGULARIZATION-IMAGES-SD. Where shall I put them? Feels like something is missing

• If you mean "Train_text_encoder_for" it was 100. Did not touch it.

[ykurilov]

Train_text_encoder_for when it's 100, I have the same result. Try to lower it to 50. That's what I'm trying to do now. I've also tried to make it 10% and the result was ok.

[akilmaf]

Yeah %50 works just fine. What I realised is that there should be at least 50 photos taken in different places & backgrounds. This way the training will be much better.

[TheLastBen]

Hi,
What did you name your instance images ?

[brian-tam]

Hello Akilmaf,

How do you use "person" and "man"?
I couldn't find the option to change the class and instance name

[nawnie]

this one uses a different system then shivam, you name your files what you want the instance to be. i have been toying with using file names as my instance prompts with good results, others have tried this as well. here is a discussion where i'm outlining what i'm doing, no examples just yet tho. working on a comparison project now.

#427

[brian-tam]

Thank you nawnie. Does naming my files as instance prompts train the "person"/"man" class in the model?

[nawnie]

for all intense and purposes class does not exist, in this repo and only subject is defined.

class is a term used when using images to compare the new subject too. so if you class something a man you show it many pictures of men so it can then compare what in your subject picture is similar to class this is why it was recommended you take different poses in different backgrounds so both class and your images have those variables changing each time so only what was unique about the subject needed to be recorded, this however uses your caption (filename) to replace the caption in the in the SD data set.

if you need to use real names you are going to increase the need for training time, one thing i mentioned yesterday is when using the name whitney for a friend i got whitney houston with whitney my friends hair. we can counter this by using long unique prompts for file names, but then each word will call to that subject, so say my subject images have one labeled, whitney a petite girl wearing glasses blows out her birthday candles.

if i then try to do a prompt whitney drives her car, well whitney houston is very well represented in the sd model so ill get a picture of her, but if i then type whitney drives her car, wearing glasses on her birthday. and i'm more likely to get my subject image.
** if i use more commo spelling whittney this effect is less pronounced but can still blend with the original info

TL:DR there is no class and i extended my reply to your other post here, ie dont use real names if you do use the most common spelling of it.

[brian-tam]

ty for the detailed reply nawnie! It was very helpful. I tried to make the filename as unique as I could like (tttwwwhhhbbb.jpg). I typed prompt like photo portrait of tttwwwhhhbbb, greek god, white hair, (and added some adjective), but it generates a similar style and the generated face does not look like me. Do you why is it the case?

[TheLastBen]

make sure you check the box "update_repo"

[akilmaf]

make sure you check the box "update_repo"

yeah it was checked.
What do you think that causes same photo generation? the generated images always come with the buildings behind me in the original photos. Even though I had other photos with blank background

[nawnie]

What are your settings for dreambooth, also have you tried adjusting the cfg scale at bottom of ui or using (((name))). Each parentheses adds weight to what's in it.

…

On Fri, Nov 25, 2022, 4:04 PM Brian Tam ***@***.***> wrote: ty for the detailed reply nawnie! It was very helpful. I tried to make the filename as unique as I could like (tttwwwhhhbbb.jpg). I typed prompt like photo portrait of tttwwwhhhbbb, greek god, white hair, (and added some adjective), but it generates a similar style and the generated face does not look like me. Do you why is it the case? — Reply to this email directly, view it on GitHub <#538 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZPYLSCO64TZUS2CPSS6FC3WKEZXPANCNFSM6AAAAAASE3RDZE> . You are receiving this because you commented.Message ID: <TheLastBen/fast-stable-diffusion/repo-discussions/538/comments/4240080@ github.com>

[brian-tam]

Sure! It looks like changing the text_encoder is essential to the style. I’ll try again. I got another question. I uploaded the trained model (my session) to hugging face, and loaded it with my api key, created the new session for training, but it asked me if I wanna overwrite an existing model or train a new model. But how do I fine tune a trained model? I would like to train the model with more data sets. Thank you!!

[nawnie]

Working on a very detailed guide with examples i hope to be up next week, also on my fork im about to try and put a 2 or 3 click dreambooth, bassicly you tell me how many subjects and ill have an option to use a recommended settings

will be LastBens instructions and if my test continue well ill include mine as a preset as well if it works well once Bens 2.0 is out ill attempt to merge or just ask as a separate easyBooth

[nawnie]

load the same session make sure resume is on and type no to resume or yes to start over

when i say to train a style i mean to train something like art in the style of Davinci. a person at 10% is doable with alot of extra steps, just a rule of thumb it seems no matter how many images at about 1k steps the encoder is done, i could be quitting to early in these trials but ill find out soon.

so maybe try training it for 100% for 350 steps then train until it looks good to you start at 200 per image

[brian-tam]

Thanks for the tip nawnie. This is very helpful! I just tried 100% for 350 steps, it produces promising results! If I overtrained my model at some point, do I have to start over to train the original model again? I now know how to fine-tune the model by using resume training. But if I overtrained it, what should I do? Thank you so much!!

[brian-tam]

I have 151 pictures in total, if I am training it at 100%, should I still use 200 steps per image?