The control images correspond one-to-one with each other, but the corresponding control image still cannot be found #677
Description
My toml file:
[general]
resolution = [1024, 1024] # optional, [W, H], default is [960, 544]. This is the default resolution for all datasets
caption_extension = ".txt" # optional, default is None. This is the default caption extension for all datasets
batch_size = 1 # optional, default is 1. This is the default batch size for all datasets
num_repeats = 1 # optional, default is 1. Number of times to repeat the dataset. Useful to balance the multiple datasets with different sizes.
enable_bucket = true # optional, default is false. Enable bucketing for datasets
bucket_no_upscale = false # optional, default is false. Disable upscaling for bucketing. Ignored if enable_bucket is false
[[datasets]]
image_directory = "data/image1"
caption_extension = ".txt" # required for caption text files, if general caption extension is not set
resolution = [1024, 1024] # required if general resolution is not set
batch_size = 1 # optional, overwrite the default batch size
num_repeats = 1 # optional, overwrite the default num_repeats
enable_bucket = true # optional, overwrite the default bucketing setting
bucket_no_upscale = true # optional, overwrite the default bucketing setting
cache_directory = "cache" # optional, default is None to use the same directory as the image directory. NOTE: caching is always enabled
control_directory = "/data/testimage3" # optional, required for dataset with control images
My dataset:
when i run :
python src/musubi_tuner/qwen_image_cache_latents.py
--dataset_config data/test.toml
--vae models/diffusion_pytorch_model.safetensors
still raise ValueError :
--dataset_config data/test.toml
--vae models/diffusion_pytorch_model.safetensors
Trying to import sageattention
Failed to import sageattention
INFO:main:Load dataset config from data/test.toml
INFO:musubi_tuner.dataset.image_video_dataset:glob images in data/image1
INFO:musubi_tuner.dataset.image_video_dataset:found 14 images
INFO:musubi_tuner.dataset.image_video_dataset:glob control images in /data/testimage3
INFO:musubi_tuner.dataset.image_video_dataset:found 0 matching control images for 1 images
ERROR:musubi_tuner.dataset.image_video_dataset:Could not find matching control images for 14 images: {'data/image1/7.jpg', 'data/image1/13.jpg', 'data/image1/12.png', 'data/image1/1.png', 'data/image1/3.png', 'data/image1/9.png', 'data/image1/10.png', 'data/image1/6.jpg', 'data/image1/2.png', 'data/image1/5.jpg', 'data/image1/8.jpg', 'data/image1/11.png', 'data/image1/4.png', 'data/image1/14.jpg'}
Traceback (most recent call last):
File "/root/musubi-tuner/src/musubi_tuner/qwen_image_cache_latents.py", line 161, in
main()
File "/root/musubi-tuner/src/musubi_tuner/qwen_image_cache_latents.py", line 136, in main
train_dataset_group = config_utils.generate_dataset_group_by_blueprint(blueprint.dataset_group)
File "/root/musubi-tuner/src/musubi_tuner/dataset/config_utils.py", line 281, in generate_dataset_group_by_blueprint
dataset = dataset_klass(**asdict(dataset_blueprint.params))
File "/root/musubi-tuner/src/musubi_tuner/dataset/image_video_dataset.py", line 1551, in init
self.datasource = ImageDirectoryDatasource(
File "/root/musubi-tuner/src/musubi_tuner/dataset/image_video_dataset.py", line 901, in init
raise ValueError(f"Could not find matching control images for {missing_controls} images")
ValueError: Could not find matching control images for 14 images
I really have no idea what's going on here