Modify / add section on random variables; make random variables central to the Baysian modeling (insted of distributions)

[amal-ghamdi]

• Make decision to change to entire book to use random variables or only introduce a section for random variables and illustrate using it in Gibbs (discussion with the team)
• If we want to change the entire book presentation to use random variables instead of distributions, we need to modify sections where we define Bayesian models and distributions for the model to follow something like https://cuqi-dtu.github.io/CUQIpy/user/_auto_howtos/RandomVariablesAndAlgebra.html#sphx-glr-user-auto-howtos-randomvariablesandalgebra-py and make sure the change is consistent throughout
• Make decision about the distribution notebook in the book, how to update the presentation to talk about (both/one of) distributions and random variables
• Add section somewhere about algebra of random variables, what are the limitations, and what do we mean by vector operations (it is Component-wise operation we believe). Make it clear that random variables are either scalars or vectors (not matrices)
• The discussion about Random fields, do we need update there.
• make sure modifying the pyTorch example with random variables works as expected.
• illustrate two ways of creating random variables
• make sure to inform users that we still able to use lambda functions (possibly in distribution notebook or somewhere else)
• can implicit prior section be consistent with the Random variable-based presentation? if not? is this confusing for the reader.. to use random variables in some but not all cases
• (to keep in mind) Need to change docstrings and examples and documentation demos to use RandomVariables for consistency

Bullets shown at our group meeting

1. What is random variable in CUQIpy?
   • A representation of random variable concept in statistics which follows a particular distribution and we can apply algebra on.
   • So roughly distribution combined with algebra, e.g., addition, subtraction, and power.
2. What’s the current status with distributions and random variables in CUQIpy?
   • Currently, all code demos, tests, how-to, and documentations in CUQIpy and CUQI-book are using distributions consistently.
   • Random varialbe still lives in an experimental module of CUQIpy, though we’re just in the process of moving it to more suitable places.
   • We have one notebook in CUQIpy’s documentation showing the use of random variables.
3. What are the benefits of using random variables?
   • The (sole?) benefit of using random variables is that with random variables, we don’t rely on lambda functions when specifying hierarchical priors (priors with hyper parameters). A comparison (based on the one from CUQIpy’s homepage); Note the extra .rv and removed lambda:

d = Gamma(1, 1e-4)
s = Gamma(1, 1e-4)
x = LMRF(0, lambda d: 1/d, geometry=A.domain_geometry)
y = Gaussian(A@x, lambda s: 1/s)

d = Gamma(1, 1e-4).rv
s = Gamma(1, 1e-4).rv
x = LMRF(0, 1/d, geometry=A.domain_geometry).rv
y = Gaussian(A@x, 1/s).rv

4. What might be obstacles when migrating fully from distributions to random variables?
   • How we present implicit priors, including those, like constrained Gaussian, with regularized linear RTO and those, like Moreau-Yoshida, Tweedie, and Restoration for MYULA and PnPULA, don’t seem to fit how we present random variables.
   • The above might introduce confusion to a user.
   • A full migration at CUQIpy’s coding side might cause some/a lot of time.
5. What to do/write in CUQI-book?
   • Option 1: migrate to random variable in general but don’t use it when talking about implicit priors
   • Option 2: stick to distributions but mention the possible use of random variable before or at the Gibbs examples
   • Option 3...