Does Musubi discard buckets with 1 or 2 images?

[Oruli]

Is this an option? I have quite a lot of images being put into their own bucket, when it would be much more preferable to crop them and put into a larger bucket, especially with larger batch size, is that right?

My dataset is 500 images

INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1008, 1040, 9), count: 1
INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1024, 1024, 9), count: 3
INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1040, 1008, 9), count: 2
INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1072, 976, 9), count: 2
INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1104, 944, 9), count: 2
INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1120, 928, 9), count: 1
INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1136, 912, 9), count: 1
INFO:musubi_tuner.dataset.image_video_dataset:bucket: (1168, 896, 9), count: 1

[kohya-ss]

Musubi Tuner will train batches that are smaller than the batch size as they are. That is, even if the batch size is 4, there will be some batches with 1, 2, or 3 images.

This is a matter of opinion, but we think it's better than simply duplicating or dropping.

One option is to set num_repeats to the same as the batch size. This will ensure that all batches have the same number of images as the batch size.

[Oruli]

It would be very handy to either

1. Force a smaller number of buckets, meaning images will be more harshly cropped and make it into bigger buckets
2. Specify bucket sizes, if that isn't already possible? a lot more work going through a large dataset though to do the bucketing manually
3. Just give the user an option to crop harder fitting the images into more large buckets

Again diffusion-pipe does this, the same dataset I only had 3 dropped images, here there about 20-25 in their own bucket which cannot train well