[Bug]: Inconsistent results from `LinearMCObjective`

[AdrianSosic]

What happened?

I'm currently playing around with mechanisms for function minimization as suggested here and noticed some strange behavior. Potentially I'm overlooking some crucial aspect or maybe there is simply a flaw in my approach. But what I observe is that the acquisition values obtained when using a LinearMCObjective do not match up when maximizing a GP or minimizing an equivalent GP trained on the negated target values.

Please provide a minimal, reproducible example of the unexpected behavior.

Here minimal example inspired by the code from the landing page. With qProbabilityOfImprovement, the differences are even more striking.

import random

import numpy as np
import torch
from botorch.acquisition import qLogExpectedImprovement
from botorch.acquisition.objective import LinearMCObjective
from botorch.fit import fit_gpytorch_mll
from botorch.models import SingleTaskGP
from botorch.models.transforms import Normalize, Standardize
from gpytorch.mlls import ExactMarginalLogLikelihood

train_X = torch.rand(5, 2, dtype=torch.double) * 2
Y = 1 - torch.linalg.norm(train_X - 0.5, dim=-1, keepdim=True)
Y = Y + 0.1 * torch.randn_like(Y)


def fix_seed(seed=0):
    torch.manual_seed(seed)
    random.seed(seed)
    np.random.seed(seed)


def run(minimize: bool):
    if minimize:
        train_Y = -Y
        obj = LinearMCObjective(weights=torch.tensor([-1.0]))
        best_f = -Y.min()
    else:
        train_Y = Y
        obj = LinearMCObjective(weights=torch.tensor([1.0]))
        best_f = Y.max()

    gp = SingleTaskGP(
        train_X=train_X,
        train_Y=train_Y,
        input_transform=Normalize(d=2),
        outcome_transform=Standardize(m=1),
    )
    mll = ExactMarginalLogLikelihood(gp.likelihood, gp)

    fix_seed(0)
    fit_gpytorch_mll(mll)
    posterior = gp(train_X)

    acqf = qLogExpectedImprovement(model=gp, best_f=best_f, objective=obj)

    fix_seed(0)
    acqf_value = acqf(train_X.unsqueeze(1))
    return posterior, acqf_value


posterior_max, acqf_max = run(False)
posterior_min, acqf_min = run(True)

# The predictions are equivalent
assert torch.equal(posterior_max.mean, -posterior_min.mean)
assert torch.equal(posterior_max.stddev, posterior_min.stddev)

# But the acquisition values do not match
print(acqf_max.tolist())
print(acqf_min.tolist())

Sometimes, the number "sort of match up" like:

[-41.65546159698638, -44.19458355943439, -5.794877518131068, -3.9572762719339836, -43.939276772549725]
[-41.54524809215892, -44.165510927608004, -5.034962168263423, -3.5053637720690194, -43.90618926555327]

But oftentimes, there are very significant differences:

[-41.253846088273036, -42.4522252849054, -43.566397044168006, -4.586150277498735, -43.92969884779241]
[-5.776667325514115, -41.053594326714006, -42.89699038185324, -1.3871302702025305, -43.38882095231844]

Is something wrong with my logic?

Please paste any relevant traceback/logs produced by the example provided.

BoTorch Version

0.12.0

Python Version

3.10.14

Operating System

macOS

Code of Conduct

• I agree to follow BoTorch's Code of Conduct

[saitcakmak]

Hi @AdrianSosic. The issue here is best_f = -Y.min(). best_f is compared to objective values, so it should be obj(train_Y).max() = Y.max() even in the minimize case.

[AdrianSosic]

Hi @saitcakmak, thanks for the quick response. I do have my trouble understanding the logic behind, though. These are my concerns:

1. First a very objective observation: If I use Y.max() also in the minimize case, the numbers are
   • generally "closer" than in the original code and in most cases there is no big difference
   • but if you run it several times, you'll still see a drastic gap somtimes. Here the outcome of one such run (see second entry):

     [-44.31642632073237, -39.24304065889777, -43.412800363898974, -42.622768652200996, -4.257021584505255]
     [-44.31624660730143, -10.235137206528035, -43.412491991884835, -42.62219805342741, -4.2532742249170665]

2. Even though the numbers are closer, I expect that there should be an exact match when all random seeds are properly controlled. Assuming Y.max() is the correct thing to do, what would explain the numerical differences?
3. Just conceptually, I don't understand how Y.max() could possibly be the right choice 🤔 If we talk about function minimization using expected improvement, then the minimum of my data is what literally defines the baseline to beat, i.e. Y.min() is the thing that really matters. (Alternatively, one could use the minimum of the posterior mean, but then we wouldn't need to touch the training data at all here). If I use Y.max() instead, the minimum doesn't enter the computation at all!?!?

[saitcakmak]

if you run it several times, you'll still see a drastic gap somtimes

I don't have a good explanation for this. I ran it 10+ times and couldn't produce a meaningful difference. Just speculating, the GPs you're using to evaluate the two cases are not identical. They're re-fitted to negated data. It's possible that there are some marginal differences that led to a big difference in acquisition values in an edge case. Rather than re-fitting the model, you could consider re-using the same statedict from one model to eliminate any differences due to model fitting. Even then, we can't eliminate all possible numerical issues, so it will still be possible to get somewhat different results.

there should be an exact match

In general, do not expect exact match when using BoTorch. The seeds only control the RNG. There is inherent randomness in low level numerical (linalg) operations we use from PyTorch. It is near impossible to make it behave fully deterministically. Setting seeds should generally get you pretty close but there will often be slight numerical differences.

function minimization using expected improvement

I think this is the source of confusion here. There is no minimization. We're maximizing the objective using EI. If your objective is -Y, then it is equivalent to minimizing Y. Internally, everything is based on maximizing self.objective(samples) and this is what the best_f should be comparable to.

In this case, the objective is defined as obj(-Y) = Y. If we provide best_f = obj(-Y).max(), this becomes best_f = Y.max().