Modelling for discontinuity in solution space

[sambitmishra98]

Is there a way to model when I know that there is at least one point of discontinuity in the solution? I have been modifying covar_module in SingleTaskGP but that doesn't seem to work so well. Based on the graph, the minima is most likely around 18-20 units. But the PosteriorMean acquisition function will tell me that its near 22.

[saitcakmak]

You could probably get something working with a custom mean_module. If you had two mean modules, mean_1 and mean_2 and implemented a new mean module that returned mean_1(X) if X < threshold else mean_2(X) (you'll need to be a bit more clever here to make this work for batch evaluation), it'd probably get you better results than the covar module. At the end of the day, the posterior mean is just the prior mean + some update term.

[sambitmishra98]

I would need to know the location of discontinuity for this?

[saitcakmak]

If you register threshold as a parameter (self.register_parameter(threshold) on a torch Module - threshold has to be a tensor), then you can learn it with all the other hyper parameters. You may want to add a prior as well if you have a rough idea of where the location is. I'd recommend checking out the ConstantMean implementation: https://docs.gpytorch.ai/en/stable/_modules/gpytorch/means/constant_mean.html#ConstantMean

[sambitmishra98]

I may need to invest more time to figure this out. I will try restricting my bounds instead.

[sdaulton]

Input warping might also be quite helpful here. See https://botorch.org/tutorials/bo_with_warped_gp

[sambitmishra98]

Thanks, I will check it out.

[sambitmishra98]

This is really interesting, but not quite useful for the problem I am solving. I showed a simplified 1-D version of a 4-D code which has a low cost function in a part of the domain and NaN value near some of the boundaries of the domain.

[sambitmishra98]

Going one step back, I had assigned a large number to the costs towards the left of the graph because they were NaNs. Let's say I instead wish to have the points with cost above 40 to be excluded from the domain. I can use inequality constraints and exclude those points right?