Train text encoder

[raulc0399]

added code to train the text encoder, with options for lora rank, learning rate and early stopping

[linnanwang]

@raulc0399 just to be curious, why would you want to train text encoders?

[raulc0399]

@linnanwang there are several papers that explore training the text encoder for sdxl. so i thought to try it with pixart - but this trains the textencoder it is no reproduction of any of those papers.

also some posts on reddit show some improvements while training for the text encoder for sdxl: https://www.reddit.com/r/StableDiffusion/comments/17xdb3m/text_encoder_onoff/

my tests show that with smaller data sets it converges sooner and it improves the quality of the generation.
also if using the prompts without the trigger words, the generation was closer to the original basemodel.
but i did run only a few numbers of tests ... but would be interesting to see what others find out

i will also try it with some regularization images to try to reproduce dreambooth training.