Sampling methods beyond non-uniform distributions

[ngeiswei]

Currently we have 3 uniform samplers in MeTTa

random-int
random-float
flip (which is temporary, and should probably be replaced by #779)

It would be nice if MeTTa had access to a standard-like library offering more sophisticated samplers based on non-uniform distributions such as Gaussian, Gamma and Beta. These samplers do not have to be in the standard library per say (as in stdlib.metta, BTW, maybe it should be renamed prelude.metta, like in Haskell, meaning it is imported in every MeTTa program by default), but it should be easy to access and part of a "standard library of modules".

One can of course do non-uniform sampling via Python, but given the AI centric nature of MeTTa, I believe it makes sense to have that
directly in a standard module.

BTW, such module could offer more than sampling, one could for instance get the mean, the mode, the standard deviation or any standard statistics defined for a particular distribution. A great example of probability distribution module can be found in the C++ Distribution STL.

[vsbogd]

I would say the only reason it is not done yet is a lack of a clear way of introducing new MeTTa modules written in Rust. Me and @besSveta currently working on improving the situation with modules in Rust. I hope we will be able introduce new modules in a nearest future.

[Necr0x0Der]

I would say the only reason it is not done yet is a lack of a clear way of introducing new MeTTa modules written in Rust

That's right. Although this is possibly not the only reason. Also, we may want to have a random number generator library with specific features like the possibility to keep track of the last samples in order to do resampling. In general, I'm not sure if we need to reimplement a rich RNG library while it is available through py-dot/py-atom, if we don't want to add some specific features there. I'm inclined towards the following:

• to have a rudimentary basic RNG library in Rust for the most convenient use in ordinary programming
• to use numpy.random or scipy.stats (or whatever else) via py-dot for ordinary machine learning / data analysis / scientific computations
• to implement a specific RNG library for probabilistic programming with distributions and samplers, which are not just black-box imperative functions

[ngeiswei]

In case one uses numpy, I assume that the random generator will be distinct from &rng or anything that's been obtained from new-random-generator. Likewise, setting the random seed will be done differently than calling set-random-seed, correct? This is not a big deal of course, one just needs to be aware of it.

[Necr0x0Der]

Yes, this is correct.

[ngeiswei]

I am contemplating adding Beta distribution in random.metta by leveraging rand_distr. I suppose I would need to edit random.rs, come up with a MeTTa API for manipulating distributions and expose this API in random.metta. I wonder though, do you think I can have everything in one file, random.metta? Or should I create a module hierarchy like

random.metta
    / \
     |
distribution.metta
    / \
     |
beta_distribution.metta
...

?

[vsbogd]

I don't see a big deal in migrating from one module to a few modules, thus I would start from making a single module and then if it will be big enough, split it on few modules.