not use gan loss?

[sunjian2015]

By train_vae.py LPIPSWithDiscriminator3D was trained, but in the file train.sh, parameter disc_start is set 9999999, does this mean that gan loss is not used or train steps is bigger than 9999999?

[ZouYa99]

Hi,
apologies for the misunderstanding caused by the script. :)
The paper mentions a two-stage training strategy: in the first stage, GAN loss is excluded for training stability, and it is added only after the model has nearly converged. Here, setting disc_start to 9999999 is intended for the first stage of training. Since it is difficult to precisely determine the exact step at which each model converges during the first stage in actual training, I adopted this setting for convenience. Once I observe that the loss and evaluation metrics have stabilized, I modify disc_start to 0 and resume training for the second stage.

[sunjian2015]

Hi, apologies for the misunderstanding caused by the script. :) The paper mentions a two-stage training strategy: in the first stage, GAN loss is excluded for training stability, and it is added only after the model has nearly converged. Here, setting disc_start to 9999999 is intended for the first stage of training. Since it is difficult to precisely determine the exact step at which each model converges during the first stage in actual training, I adopted this setting for convenience. Once I observe that the loss and evaluation metrics have stabilized, I modify disc_start to 0 and resume training for the second stage.

ok, thanks

[sunjian2015]

Hi, apologies for the misunderstanding caused by the script. :) The paper mentions a two-stage training strategy: in the first stage, GAN loss is excluded for training stability, and it is added only after the model has nearly converged. Here, setting disc_start to 9999999 is intended for the first stage of training. Since it is difficult to precisely determine the exact step at which each model converges during the first stage in actual training, I adopted this setting for convenience. Once I observe that the loss and evaluation metrics have stabilized, I modify disc_start to 0 and resume training for the second stage.

hi, I add gan loss after training stable, but after that, continue training both SSIM and PSNR are declining, is this right?

[k15201363625]

Hi, apologies for the misunderstanding caused by the script. :) The paper mentions a two-stage training strategy: in the first stage, GAN loss is excluded for training stability, and it is added only after the model has nearly converged. Here, setting disc_start to 9999999 is intended for the first stage of training. Since it is difficult to precisely determine the exact step at which each model converges during the first stage in actual training, I adopted this setting for convenience. Once I observe that the loss and evaluation metrics have stabilized, I modify disc_start to 0 and resume training for the second stage.

hi, I add gan loss after training stable, but after that, continue training both SSIM and PSNR are declining, is this right?

Same question. and I try to set different learning rate, the ssim and psnr are decreasing, only lpips become better. Furthermore, the reproduced results has large gap with released checkpoint.

[ZouYa99]

Yes, after incorporating the GAN loss in the second stage, PSNR and SSIM might slightly decrease, but LPIPS and rFVD will improve. A trade-off needs to be made when selecting the final checkpoint in experiments. :)

Additionally, the checkpoints I released represent the best results achieved after training for a comparatively longer duration. During the VAE training process, we observed that metrics consistently improve as training time increases, even though the loss decreases relatively slowly. If you aim for a better trained model, you could try balancing the batch size and learning rate, as well as extending the training time.