Do I have to use a data set like SAM？

[zmy-08]

I really enjoyed this project, but I had this problem when I was training my own data set, and my data set was formatted like this: Do I have to use a data set like SAM? Do I have to prepare myself in advance for the caption_feature_wmask and img_vae_feature?

SAM datasets：
SA1B
├──images/ (images are saved here)
│ ├──sa_xxxxx.jpg
│ ├──sa_xxxxx.jpg
│ ├──......
├──captions/ (corresponding captions are saved here, same name as images)
│ ├──sa_xxxxx.txt
│ ├──sa_xxxxx.txt
├──partition/ (all image names are stored txt file where each line is a image name)
│ ├──part0.txt
│ ├──part1.txt
│ ├──......
├──caption_feature_wmask/ (run tools/extract_caption_feature.py to generate caption T5 features, same name as images except .npz extension)
│ ├──sa_xxxxx.npz
│ ├──sa_xxxxx.npz
│ ├──......
├──img_vae_feature/ (run tools/extract_img_vae_feature.py to generate image VAE features, same name as images except .npy extension)
│ ├──train_vae_256/
│ │ ├──noflip/
│ │ │ ├──sa_xxxxx.npy
│ │ │ ├──sa_xxxxx.npy
│ │ │ ├──......

The format of my dataset is as follows：
food_test
├──InternImgs/ (images are saved here)
│ ├──xxxx1.png
│ ├──xxxx2.png
│ ├──......
├──InternData/ (corresponding captions are saved here, same name as images)
│ ├──xxxx1.txt
│ ├──xxxx2.txt

[lawrence-cj]

Do I have to use a data set like SAM?

You can have your own structure, but you will need to modify the dataset class in the project

Do I have to prepare myself in advance for the caption_feature_wmask and img_vae_feature?

Yes.

If you want to skip all these problems and train. You are welcome to try our newer code repo: PixArt-Sigma: https://github.com/PixArt-alpha/PixArt-sigma. It's much more user-friendly.