Initial Noise 是怎么统计得来

[13718413797]

Initial Noise 是怎么统计得来的 是否有相关的代码demo

[zhuhz22]

Hi @13718413797 ,
Thank you for your attention to our work! We traverse the video, passing them through the VAE to encode, then compute the mean and variance of these latents for the initial noise.

An illustrative demo is as follows (for simplicity, the specific function details are omitted) :

    cnt=0
    with torch.no_grad(), torch.cuda.amp.autocast():
        # ----------- Traverse the video dataset to obtain its latent space tensor. ---------------
        for root, dirs, files in os.walk(gt_path):
            for file in files[:args.total-1]:
                file_path = os.path.join(root, file)
                gt_video=get_gt(file_path,video_size=(args.height, args.width)).to("cuda")
                gt_video=get_latent_z(model,gt_video)
                # ---------- For the mean, simply sum each tensor element-wise and divide by the total number. -----
                # For variance, we use the formula Var(x) = E(x²) - E²(x). Therefore, we need to calculate E(x²), which is computed as gt_video**2 / sum.
                if cnt==0:
                    sum=gt_video
                    sum_for_var=gt_video**2
                else:
                    sum+=gt_video
                    sum_for_var+=gt_video**2
                cnt+=1
                
        # The mean has been calculated.
        expectation_X_0=sum/cnt
        # Here, the sum of the element-wise variances is calculated.
        tr_covar_X_0=sum_for_var/cnt-expectation_X_0**2
        tr_Cov=tr_covar_X_0.sum()
        # Divide the sum of variances by the total number of elements in the tensor.
        shape_ = expectation_X_0.shape
        d = 1
        for dim in shape_:
            d *= dim
        var=tr_Cov/d
        #------------------After calculating the mean and variance, apply the final weighting.-------------
    sqrt_alpha_t=model.get_sqrt_alpha_t_bar(gt_video,torch.tensor([args.sdedit_t-1]).to('cuda'))
    # Final mean.
    mu_p=sqrt_alpha_t*expectation_X_0
    # Final std
    alpha_t=sqrt_alpha_t**2
    sigma_p=torch.sqrt(1-alpha_t + alpha_t*var)