Combining chaopy PCE with scikit learn Gaussian Process Regression to form Polynomial chaos kriging(PCK)

[gaoutham92]

Hi,

I am using PCE in Chaospy for my research. I can see in the documentation that you have provided an interface to some regression techniques available in Scikit-learn that contain 'coef_' attribute, and use them for polynomial fitting in Chaospy. I tried them and they work perfectly, additionally, I wanted to use a feature called Gaussian process regression available in Scikit-learn and combine it with the polynomial chaos expansion in Chaospy - inorder to develop Polynomial Chaos Kriging (PCK) model. I tried to make the interface myself, but the Gaussian process regression in Scikit-learn does not have 'coef_' attribute and that makes the interface complicated. My trials & guesses to interface went in vain and gave highly erratic results. So it would be highly beneficial if you can make that interface between PCE in Chaospy and Gaussian Process Regression(GPR) in Scikit-learn (like we can use GPR to fit the polynomial coefficients in Chaospy) or at least if you could kindly provide some instructions/opinions to achieve the same it would be immensely helpful as you have more knowledge on the Chaospy package and Scikit-learn than me. Looking forward for your help.

                                               Thankyou

[jonathf]

This is an interesting idea. Thanks for biringing it to my attention.

I am not to familiar with Polynomial Chaos Kriging models. Do you have any good references to share so I can take a look at what is proposed?

[gaoutham92]

Hi,

Thanks for your response. The PCK models are implemented in UQlab (Framework for uncertainty quantification in MATLAB) from ETH Zurich - Chair of Risk, Safety, and Uncertainty Quantification. I have provided links to some useful references, examples, and UQLab manuals obtained from UQLab and ETHZurich websites below.

https://www.uqlab.com/pck-user-manual
https://www.uqlab.com/pc-kriging-introduction
https://sudret.ibk.ethz.ch/research/past-projects/polynomial-chaos-kriging.html

[jonathf]

Okay, so if I get it right (nd please correct me if I am not), PCK is follows:

Point Collocation with LARS regression (to reduce the number of PCE terms), which is then used as the "mean field" input in the Kriging estimation.

My initial thought here is that you need a Kriging method that supports custom mean fields as input. This isn't too difficult, but doesn't look like scikit-learn support that. From the docs on GaussianProcessRegression:

The prior mean is assumed to be constant and zero (for normalize_y=False) or the training data’s mean (for normalize_y=True).

In other words, you can not set it to something custom, like a PCE.

I imagine that applying a trick like modelling the distance between the data and the PCE model in Kriging is possible, but I am not familiar enough with Kriging estimation to say how coupled the mean field is the to covariance structure.

[jonathf]

Let me brew on the topic a few days, and I'll let you know if there is good way forward using chaospy or not.

[gaoutham92]

Hi
Yes, you are right, the mean-field of the Universal kriging estimation should be orthonormal polynomials and we should use the universal kriging in Polynomial chaos kriging models.

Actually, in kriging, there are many different types - Ordinary kriging, Universal kriging and many advanced types.

The Ordinary kriging uses a constant mean as trend function, so it is not applicable for PCK model, however universal kriging has a trend/mean function consisting of orthonormal polynomials and therefore it is applicable for PCK.

And I can see that Gaussian process regression in Scikit-Learn has Ordinary and Universal kriging implemented I believe, and we should use the weights estimated for universal kriging and not the ordinary kriging. I have attached a link for your reference

https://ogrisel.github.io/scikit-learn.org/sklearn-tutorial/modules/generated/sklearn.gaussian_process.GaussianProcess.html

It is not urgent and please take your time and apologies if I happen to disturb you.

[jonathf]

@gaoutham92, I forwarded your question to the PyKrige team. They might have some useful insight into what is possible.

[gaoutham92]

Hi,

Thanks for your efforts. I will follow it from them

[gaoutham92]

Hi Jonathan,
I could see the Polynomial chaos kriging functionality is available in Chaospy using the universal kriging from Gstools. Thanks for implementing the functionality at the earliest.