Carry posteriors over as custom priors for new model

[Miguel-Adarlo]

Hello, I read the documentation, and it had this line: "Note however that you can always use the posteriors from a previous modeling cycle as priors when you refresh the model."

I am trying to pass over my posteriors from an old model to another via the model's trace like so:

#Save the trace as pkl for later input into the next model.
#The trace contains the posteriors.
import pickle

with open('custom_priors.pkl', 'wb') as fp:
    pickle.dump(mmm.trace, fp)

and in another notebook:

with open('custom_priors.pkl', 'rb') as fp:
    c_priors = pickle.load(fp)

but when trying to load the posteriors as priors, I get this error.

ValueError: The following passed custom priors dont have a match in the model. Please double check the names have been written correctly: {'mu', 'coef_media', 'media_transformed'}

What have I been doing wrong in regards to using the posteriors as custom priors? Thanks in advance!

[statm3n]

Thanks for the question! I would guess the format of some priors in custom_priors.pkl do not have designed input format. I would recommend checking the distributions saved in custom_priors and also check this page on using custom priors in LightweightMMM.

[Miguel-Adarlo]

I looked in the source code, and it seems I raised a value error right around here:

The values I passed that didn't have a match were mu, coef_media, and media_transformed. Is there any way to pass those? These were all obtained from the trace of a prior MMM model.

Oh, and as an added edit, when I removed the keys, here's what I got:

ValueError: Priors given must be a Numpyro distribution or one of the following to fit in the constructor of our default Numpyro distribution. It could be given as args or kwargs as long as it is the correct format for such object. Please refer to our documentation on custom priors to know more.

And what I'm passing looks like this:

{'coef_extra_features': Array([[-1.0046250e+00,  4.0052598e-03, -4.2482973e-03, ...,
          3.2380156e-03,  3.2602430e-03,  4.4624833e-03],
        [-1.0080190e+00, -7.3947706e-03, -1.0932419e-03, ...,
         -4.5722821e-03, -4.2066353e-04,  1.1567312e-02],
        [-1.0079037e+00, -7.5299763e-03, -1.1272664e-03, ...,
         -4.9203201e-03, -7.4060942e-04,  1.1377455e-02],
        ...,
        [-9.3925858e-01, -3.3400478e-03, -7.5338562e-03, ...,
         -1.6487222e-03, -1.0643747e-04,  7.1057054e-04],
        [-9.2797351e-01, -4.1975700e-03, -7.4908584e-03, ...,
         -1.5583988e-03, -1.5971746e-03, -4.8947998e-04],
        [-8.6817116e-01, -8.6360890e-03, -8.0242641e-03, ...,
         -2.4409870e-04,  5.5178599e-03,  5.8612227e-04]], dtype=float32),
 'coef_trend': Array([[1.9167017e-04],
        [3.4442415e-05],
        [2.3470368e-04],
        ...,
        [1.1025104e-03],
        [1.4777587e-03],
        [1.3625508e-03]], dtype=float32),
 'expo_trend': Array([0.86096275, 0.71804184, 0.735518  , ..., 0.69951665, 0.7023334 ,
        0.63966906], dtype=float32),
 'gamma_seasonality': Array([[[ 1.5194422e-03,  3.0154742e-03],
         [ 4.2552100e-03,  7.1312818e-03]],
 
        [[-1.1223243e-05,  2.0479739e-03],
         [ 6.9062486e-03,  6.3372413e-03]],
 
        [[-1.1281435e-04,  2.2810025e-03],
         [ 7.2362442e-03,  6.1149560e-03]],
 
        ...,
 
        [[-2.8323459e-03,  3.8892468e-03],
         [ 1.9386703e-03,  3.2950856e-03]],
 
        [[-1.6393752e-03,  4.2632222e-03],
         [ 1.7299013e-03,  2.2787557e-03]],
 
        [[-5.4988754e-04,  6.2936367e-03],
         [ 1.4661121e-03,  4.3684719e-04]]], dtype=float32),
 'half_max_effective_concentration': Array([[2.9787445 , 2.8519754 , 0.52853185, ..., 1.1006113 , 0.41622987,
         0.2539402 ],
        [3.920835  , 0.72122073, 0.49436212, ..., 0.41740042, 1.8379505 ,
         0.46195635],
        [4.0379243 , 0.7512918 , 0.4812356 , ..., 0.424952  , 2.0191407 ,
         0.41462946],
        ...,
        [0.50110257, 1.6452546 , 1.3235208 , ..., 0.55391717, 0.5670452 ,
         0.8898682 ],
        [0.503373  , 1.6467781 , 1.2343373 , ..., 0.36114064, 0.4149495 ,
         0.8953308 ],
        [2.6501098 , 0.9292237 , 0.8545067 , ..., 0.6597134 , 1.4625281 ,
         1.3686172 ]], dtype=float32),
 'intercept': Array([[0.06250319],
        [0.0559515 ],
        [0.06116943],
        ...,
        [0.1343642 ],
        [0.14555253],
        [0.02151612]], dtype=float32),
 'lag_weight': Array([[0.9416687 , 0.82010275, 0.4096106 , ..., 0.3342321 , 0.8266394 ,
         0.95633566],
        [0.96449584, 0.9170035 , 0.8320815 , ..., 0.8439371 , 0.9530579 ,
         0.9549369 ],
        [0.96413165, 0.91547287, 0.85933065, ..., 0.82200724, 0.9525178 ,
         0.95051557],
        ...,
        [0.96477604, 0.78250337, 0.8826648 , ..., 0.6497623 , 0.2402161 ,
         0.86694735],
        [0.9517046 , 0.76670337, 0.84859794, ..., 0.5778418 , 0.25244322,
         0.90083057],
        [0.71540684, 0.6787859 , 0.6339387 , ..., 0.6420567 , 0.5334207 ,
         0.9720951 ]], dtype=float32),
 'sigma': Array([[0.00440825],
        [0.00527523],
        [0.00530611],
        ...,
        [0.00422471],
        [0.00434532],
        [0.00471151]], dtype=float32),
 'slope': Array([[1.3911084 , 0.8436842 , 0.9070687 , ..., 0.4115861 , 1.5248374 ,
         0.7323983 ],
        [2.4568977 , 3.676918  , 1.0614157 , ..., 0.40869948, 1.2469206 ,
         2.9842088 ],
        [2.3004797 , 3.5466995 , 0.9874999 , ..., 0.4415909 , 1.243768  ,
         2.997884  ],
        ...,
        [0.07155922, 0.31703126, 0.39658633, ..., 0.52560085, 0.52783763,
         0.4245492 ],
        [0.08329035, 0.40289918, 0.5266781 , ..., 0.44289586, 0.53693676,
         0.3907163 ],
        [0.19296676, 0.71438503, 3.9560373 , ..., 0.2864869 , 0.35916647,
         0.73685956]], dtype=float32)}

[Miguel-Adarlo]

Wait, rereading the custom priors and how it's structured, I understand I have to be passing the distributions along with the custom priors listed in the docs. I guess my question now would be on how to pass parameters for concentration for old posteriors. From what I understand, I can pass the loc and scale for most distributions, but for distributions like the beta distribution with no set parameters, what exactly do I pass along?