model comparison and log probability for SNPE

[sesirota]

I am running a set of models with SNPE and am having trouble finding a way to quantify their model goodness of fit (given that it's likelihood free). I am having trouble finding how to compare models, but then I noticed the DirectPosterior object has a log_prob() object, so is this something that can be used to quantify the performance (outside of using simulation based calibration / posterior predictive checks)? How is the log probability determined for SNPE, given that it's likelihood-free? And does this mean that a BIC would be a valid way to compare models that were fit using SBI, or is there some other way to quantify this? Any help would be appreciated, thank you!!

[michaeldeistler]

Hi, thanks for reaching out! I think there is a misunderstanding here: sbi tackles simulators which are "likelihood-free" (i.e., one cannot use Bayesian inference methods such as MCMC which require to evaluate the likelihood). However, many of the methods in the sbi toolbox (including SNPE) have the ability to evaluate the (estimated) posterior log-probability after training.

So yes, you are free to use the SNPE log-probability for any downstream validation or evaluation.

Hope this helps!
Michael

[sesirota]

That makes sense, thank you!!On Oct 14, 2025, at 12:16, Michael Deistler ***@***.***> wrote: Hi, thanks for reaching out! I think there is a misunderstanding here: sbi tackles simulators which are "likelihood-free. However, many of the methods in the toolbox (including SNPE) have the ability to evaluate the (estimated) posterior log-probability after training. So yes, you are free to use the SNPE log-probability for any downstream validation or evaluation. Hope this helps! Michael —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[sesirota]

Hi, So follow up question. I am getting max log probabilities above zero (0.3, 0.98,etc.). this is impossible for log probability, as it corresponds to a probability above 1… is this value a probability density? Please let me know, thanks. Best, Sloane

…

On Oct 14, 2025, at 12:16, Michael Deistler ***@***.***> wrote: Hi, thanks for reaching out! I think there is a misunderstanding here: sbi tackles simulators which are "likelihood-free. However, many of the methods in the toolbox (including SNPE) have the ability to evaluate the (estimated) posterior log-probability after training. So yes, you are free to use the SNPE log-probability for any downstream validation or evaluation. Hope this helps! Michael — Reply to this email directly, view it on GitHub <#1675 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A7TXFVPXCEKV722FYPIIRUL3XUOVLAVCNFSM6AAAAACJE2VUN2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTINRXHAYTONI>. You are receiving this because you authored the thread.

[michaeldeistler]

For continuous parameters, the log_prob returns the log-likelihood of the probability density function.

[sesirota]

I see, that makes sense. So is it assumed that it will be the log-likelihood of the probability density function then?

[janfb]

It really is the log-likelihood of the probability density function :) (of the posterior)
(given that there was enough training data, NN training converged, etc).
See, e.g., in the appendix of this paper for a proof: https://arxiv.org/abs/1711.01861