Use numpy random generators to enable pickling of random state

[jklaise]

Instead of using np.random.seed we should look at assigning a seeded random generator object to the explainer instance so that the random state can be recovered after deserializing a saved explainer.

That being said, it looks like np.random.RandomState is now legacy, we could use the default generator.

[alexcoca]

What is the difference between this proposal and simply serialising the seed property and then calling np.random.seed with the saved seed upon deserialisation?

[jklaise]

It depends on the use case and where randomness is used in the algorithm. If the seed is not used before the first explain call then it could be enough to just keep track of the seed property. However, if the explainer has been running for a while and we want to save it together with the random state this would not work as the state would be reset on calling np.random.seed. The question is if something like this is a plausible use case. Also, if randomness is used before explain, then just using the seed after deserializing would not be able to reproduce experiments that just run in the notebook (__init__, fit, explain with the same seed).

[alexcoca]

Great discussion @jklaise! I think that the most general case is that some randomness is used before explain, with the simple example being the background data summarisation in KernelShap, which relies on subsampling sometimes. In that case re-setting the seed would generate a different sequence of random numbers to what would have been generated if the explainer would not have been serialised. While running the explainer repeatedly would give repeatability in this case, this would not be sufficient to cover a use case where someone was trying to reproduce some experiments starting from a serialised explainer. I think the distinction makes sense, thanks for clarifying.