Training on MNIST

[iamtonymwt]

Hi there,

First of all, really appreciate your amazing work and code base.

However, when I followed the command python train.py --data_name mnist --channels 1 --image_size 28 --hurst 0.9 --num_aug 3 --dynamics fvp --model_channels 64 --num_res_blocks 3 --attn_resolutions 4,2 --channel_mult 1,2,4 --use_ema False --log_model_every_n 50000 --lr 1e-4 --batch_size 1024 --train_steps 50000 to train the model on MNIST, after training, the images generated are not that good.

something like this:

Is this normal and do you have any idea how to solve this?

[GabrielNobis]

Hi there,

I am very sorry for the late answer. We accidentally set the default value in args.py for dropout to 1.0 instead of 0.1
It should be fixed now. Please try again with the following args to run on MNIST:

python train.py --data_name mnist --channels 1 --image_size 28 --hurst 0.9 --num_aug 3 --dynamics fvp --model_channels 64 --num_res_blocks 3 --attn_resolutions 4,2 --channel_mult 1,2,4 --dropout 0.0 --use_ema False --log_model_every_n 50000 --lr 1e-4 --batch_size 1024 --train_steps 50000

You also find these updated arguments in the README now. Please let us know if you run into other problems.

[iamtonymwt]

Hi there,
Thanks that would be great. I'll check it later. btw I still have 2 questions:

1. Why would we set --val_check_interval to 10, rather than 100? It seems like it will decrease the training speed.
2. And also there is a --conditioning arg, what's the usage of that?

[GabrielNobis]

1. Thanks for pointing that out: --val_check_interval should be greater than 10 for long training runs. We retained it as the default value to facilitate frequent feedback but updated the argument in the README. You are right, the higher you choose --val_check_interval, the faster the training, as it does not need to sample so often during training.
2. If you set --conditioning False you will train unconditional. That means that the unet will not be conditioned on a certain class and you have no control of the generated class during sampling. Please note that you need to set --conditioning consistent during training and sampling.

[iamtonymwt]

Thanks! It helps a lot!
May I ask if the evaluation code is available? for MINIST or CIFAR10? I totally understand if it's not available yet.