Probabilities of Feasibility for Classifier based constraints in Acquisition Functions

[jduerholt]

Motivation

Using classifiers as output constraints in MC based acquisition functions is a topic discussed at least in #725 and in #2700. The current solution is to take the probabilities from the classifier and to unsigmoid them. This is a very unintuitive approach, especially as sometimes a sigmoid and sometimes a fatmoid is used. This PR introduces a new attribute for SampleReducingMCAcquisitionFunctions named probabilities_of_feasibility that expects a callable that returns a tensor holding values between zero and one, where one means feasible and zero infeasible.

Currently, it is only implemented in the abstract SampleReducingMCAcquisitionFunction using the additional attribute. As the constraints argument is just a special case of the probabilities_of_feasibility argument, where the output of the callable is not directly applied to the objective but further processed by a sigmoid or fatmoid one could also think about uniting both functionalities into one argument, and modify fat to List[bool | None] | bool that indicates if a fatmoid, a sigmoid or nothing is applied. When the user just provides a bool, it applies either a fatmoid or sigmoid for all. This approach would also have the advantage that only compute_smoothed_feasibility_indicator needs to be modified and almost nothing for the individual acqfs (besides updating the types for constraints.) Furthermore, it follows the approach that we took when we implemented individual etas for the constraints. So I would favor this one in contrast to the one actually outlined in the code ;) I am looking forward to you ideas on this.

@SebastianAment: In #2700, you mention that from your perspective the probabilities_of_feasibility would not be applied on a per sample basis as the regular constraints. Why? Also in the community notebook by @FrankWanger using the unsigmoid trick it is applied on a per sample basis. I would keep it on the per sample basis and if a classifier for some reason do not returns the probabilities on a per sample basis, it would be the task of the user to expand the tensor accordingly. What do you think?

Have you read the Contributing Guidelines on pull requests?

Yes.

Test Plan

Unit test, but not yet added due to the draft status and pending architecture choices.

cc: @Balandat