Skip to content

Train_controlnet_sdxl.py and tensorboard log images #11470

Open
@RomixERR

Description

@RomixERR

Describe the bug

When logging validation images on tensorboard, one tile is always created, even if there are several validation projects and images. I suggest making changes to the code:

for tracker in accelerator.trackers:
        if tracker.name == "tensorboard":
            i=0
            for log in image_logs:
                images = log["images"]
                validation_prompt = log["validation_prompt"]
                validation_image = log["validation_image"]

                formatted_images = [np.asarray(validation_image)]

                for image in images:
                    formatted_images.append(np.asarray(image))

                formatted_images = np.stack(formatted_images)

                tag = f"{validation_prompt}_{i}"
                tracker.writer.add_images(tag, formatted_images, step, dataformats="NHWC")
                i=i+1
            tracker.writer.flush()
        elif tracker.name == "wandb":

Reproduction

Start Train_controlnet_sdxl.py with params:

accelerate
launch
train_controlnet_sdxl_orig.py
--pretrained_model_name_or_path="model/stable-diffusion-xl-base-1.0"
--pretrained_vae_model_name_or_path="model/sdxl-vae-fp16-fix"
--output_dir="output"
--train_data_dir="fillsmall2"
--cache_dir="cachedir"
--mixed_precision="fp16"
--resolution=512
--learning_rate=1e-5
--max_train_steps=10000
--train_batch_size=1
--gradient_accumulation_steps=1
--max_grad_norm=1.0
--seed=42
--checkpointing_steps=1000
--enable_xformers_memory_efficient_attention
--use_8bit_adam
--report_to="tensorboard"
--lr_scheduler="constant_with_warmup"
--lr_warmup_steps=100
--logging_dir="logs"
--validation_image
"validation_images/1.png"
"validation_images/2.png"
"validation_images/3.png"
--validation_prompt
"red circle with blue background"
"green circle with brown background"
"blue circle with white background"
--validation_steps=10
--num_validation_images=2

Open http://localhost:6006/ (Tensorboard)

Logs

System Info

Diffusers version: 0.34.0.dev0
Torch version: 2.7.0+cu126
XFormers version: 0.0.30
Python version: 3.10.11
Windows 10

Who can help?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions