Regularization in Priors

[Aminshnn]

In my problem, I have a set of paid media, control and non-media treatment variables. To decrease the number of variables, it is possible to manually remove some of them by doing some EDAs. In frequentist statistics, we can add regularization and it helps with feature/variable selection. However, Google Meridian implements Bayesian Statistics for parameter estimation. I am wondering if we can use any regularization?
Regarding the priors, we can use some probability distributions like Laplace prior, Horseshoe prior or Spike-and-slab prior as a form of regularization in Bayesian statistics. I am wondering it Google Meridian has the flexibility to handle these priors? Ideally I want to set the prior for ROI where there is atom at 0 (with probability pi which should be updated) or it has the prior LogNormal(0.2,0.9) (with probability 1-pi) or in the same way, I like to set prior for the contributions, that is, Spike-and-slab prior. Since having atom at 0 makes the distribution non-differentiable, I think we can replace it with a continuous function like logit, or any other function.

If it is possible to set such priors, how can I do that?
Thanks

[dattatreyam23]

Hi @Aminshnn ,

Thank you for contacting us!

Priors can and certainly should be used for regularization. Almost any continuous distribution can be used as long as the prior support range is within the appropriate range for each parameter. (You cannot have a negative residual standard deviation sigma, for example.)

Meridian uses Hamiltonian Monte Carlo (specifically, the No U-turn algorithm) for MCMC posterior sampling. Distributions with point mass are not supported by this algorithm. The only exception is the Deterministic distribution, which puts 100% probability mass at a single point.

Priors are set using a PriorDistribution object, which is passed to the `ModelSpec. For example, you can customize the the ROI prior in media channels like this:

PriorDistribution(roi_m=tfp.distributions.Gamma(0.5, 0.25))

To assign a different distribution on each channel, you can pass vector arguments. For example, if you have 3 media channels, you could pass a distribution like this:

PriorDistribution(roi_m=tfp.distributions.Gamma([0.4, 0.5, 0.6], [0.2, 0.25, 0.3]))

Note that if you specify an ROI prior that allows negative values, you will need to set media_effects_dist='normal' in the ModelSpec. However, allowing negative values in ROI priors is generally not recommended because this can inflate the posterior variance as opposed to regularizing it.

Feel free to reach out to us with any further queries regarding the same.

Thank you,

Google Meridian Support Team

[Aminshnn]

Thanks for your response. I am still confused how can I set priors which work as regularization. I understand that it is not possible to set priors like Spike-and-slab prior because it has probability mass at 0. But, I am wondering if it is possible to replace it with a continuous variables? I am wondering if it is possible to set the prior sigmoid*LogNormal where sigmoind works as an approximation of Bernoulli random variable? Or is it possible to set the prior in form of Beta(a,b)*LogNormal?

[dattatreyam23]

Hi,

Almost any distribution family can be used for regularization, as long as it has a parameter to control the standard deviation. By setting the standard deviation small enough, you will regularize the corresponding parameter toward some value.

The Log-Normal distribution is recommended for ROI priors because it has two parameters, which allow you to control both the mean and standard deviation. Consider using the prior_distribution.lognormal_dist_from_mean_std function to create a Log-Normal distribution with the desired mean and standard deviation. Here is an example.

Now coming to your query about setting sigmoid*LogNormal prior, there may be a way using Tensorflow bijectors to achieve what you are trying to do. However, it's generally recommended to choose a more straightforward parametric distribution from those available. Complex distributions created with bijectors could slow down the sampling or cause convergence issues.

Feel free to reach out to us with any further queries regarding the same.

Thank you,

Google Meridian Support Team