LCM and ICM

[ToennisStef]

Hi
While reading through the provided literature on the implementation of the MultiTaskGP:
[Bonilla2007MTGP]
E. Bonilla, K. Chai and C. Williams. Multi-task Gaussian Process Prediction. Advances in Neural Information Processing Systems 20, NeurIPS 2007.

I came across this line in the related work section:
"Our model for the observations corresponds to an LCM model with two processes: the process for f· and the noise process."

So i was wondering. Is the MultiTaskGP then a LCM or an ICM as stated in the documentation. It is an ICM right?
it cannot be written as:

Linear model of co-regionalization:
$f_i (x)=∑^Q_{q=1}a_{iq}∙u_{q} (x) $
Where $f_i (x)$ is the $i$-th output, $u_q (x)$ is the $q$-th latent GP, $a_{iq}$ are coefficients that define the contribution of the $q$-th GP to the $i$-th task.

I hope you can help me in this matter. I am thankful for anyone clarifying.
Best regards,
Stefan Tönnis

[saitcakmak]

Hi @ToennisStef. The MultiTaskGP utilizes an ICM kernel, as stated in it's documentation. The base covariance and task covariances are combined using an element-wise product: https://github.com/pytorch/botorch/blob/main/botorch/models/multitask.py#L308-L312

[ToennisStef]

Ok thank you very much for clarifying! There is then no implementation of the LCM in botorch?

[saitcakmak]

No, we don't have an implementation of the LCM model. We have a tutorial that implements RGPE, which is somewhat similar to LCM: https://botorch.org/tutorials/meta_learning_with_rgpe

[Balandat]

We had played around with more general LCM models in the past, but we don't have any in OSS.

So i was wondering. Is the MultiTaskGP then a LCM or an ICM as stated in the documentation. It is an ICM right? It cannot be written as:

Technically, any ICM model is also a (degenerate) LCM model with a single coefficient. So it can be written as such but it's very uninteresting as Q=1

[ToennisStef]

Thanks for the response!
I saw there is an example also in gpytorch:

https://docs.gpytorch.ai/en/v1.13/examples/04_Variational_and_Approximate_GPs/SVGP_Multitask_GP_Regression.html

Maybe i can incorporate this into botorch with the model wrapper. Interestingly, there the LCM model is defined as variational model. Need to read more into it but intuatively i would assume one could exact inference due to the linear dependencies of the latent GP's.