Question about tutorial of multi-objective BO

[A-ep93]

Hello,
I am analyzing the multi-objective BO tutorial posted in this link.
I have two questions about function optimize_qparego_and_get_observation:
This is how acquisition function is defined in this code:

weights = sample_simplex(problem.num_objectives, **tkwargs).squeeze()
objective = GenericMCObjective(get_chebyshev_scalarization(weights=weights, Y=train_obj))
acq_func = qExpectedImprovement(  # pyre-ignore: [28]
            model=model,
            objective=objective,
            best_f=objective(train_obj).max(),
            sampler=sampler,
        )

My first question: We have two outputs in this example. This means that train_obj is a 6*2 matrix. However, when we use the code best_f=objective(train_obj).max(), it only returns one maximum. Shouldn't it return one maximum for each column of train_obj? I mean, we have two outputs, shouldn't we have two maximum values here as well?

My second question is about usage of the objective function here. I cannot understand why do we need to define and use the objective function the way it appears in the code above? I assume the main objective function is the output of the black box function that we have. And here we have two outputs that represent two objective functions that need to be optimized. Then why do we need to use GenericMCObjective here?

Any ideas/comments are greatly appreciated.

[Balandat]

So the answer to your first question follows from the one to your second question - This particular part of the code uses qParEGO (https://ieeexplore.ieee.org/document/1583627) - which is really just a fancy way of saying that at each step we're picking a random scalarization fo the objectives and optimize that scalarization to generate the next candidate. By doing so you can explore the Pareto frontier.

Here the GenericMCObjective is just a particular random scalarization (namely, a Chebyshev scalarization). So the objective here will take in a multi-(in this case 2)-dimensional objective and return a scalar value. This is why best_f=objective(train_obj).max() is indeed a scalar - it is the maximum of the randomized scalarization across the previously observed outcomes. So "objective" here is used in terms of computing a derived objective on the outcomes that are observed from the black box function.

[A-ep93]

Thanks for your response @Balandat. So I think I should use qExpectedHypervolumeImprovement as it appears in the tutorial. We should use a reference point in this function. What is this reference point and how should we know it for a black-box function?

[Balandat]

I would recommend using qNoisyExpectedHypervolume instead (we just updated the tutorial with that recently: https://github.com/pytorch/botorch/blob/main/tutorials/multi_objective_bo.ipynb, it has yet to be updated on the website).

Regarding the reference point, do you have a rough understanding of the ranges the outputs can take on? Generally the reference point encodes the limits of the outcomes beyond which you don't want to keep exploring. If you don't have that kind of domain knowledge a typical heuristic that is used is to run some random configurations and then choose the nadir point (the element-wise worst objective value) and start with that reference point (and then potentially adjust it down the road as you've gained a better understanding of the general shape of the Pareto frontier.

[A-ep93]

Thank you very much for your explanations and for the useful tutorial you shared @Balandat.

[sdaulton]

a typical heuristic that is used is to run some random configurations and then choose the nadir point (the element-wise worst objective value) and start with that reference point (and then potentially adjust it down the road as you've gained a better understanding of the general shape of the Pareto frontier.

A slightly better heuristic than the nadir is implemented in infer_reference_point (https://github.com/pytorch/botorch/blob/e23ad502629846bdc0a75a2e2af7f50982a330eb/botorch/utils/multi_objective/hypervolume.py). This uses a scaled nadir point based on the range of each objective. This means that points on the end of the Pareto frontier have positive hypervolume contributions. If the nadir point is used as the reference point, then the end points of the Pareto frontier have no hypervolume contribution and the optimization will not try to learn the pareto frontier beyond the original end points of the Pareto frontier

[A-ep93]

Thank you for your suggestion @sdaulton. Can you please let me know how I can use this feature in the tutorial. I mean which parts of the tutorial should change and how?