[FEATURE REQUEST]: Extend `project_to_feasibile_space_via_slsqp` to nonlinear constraints

[jduerholt]

Motivation

I think it would make sense to extend project_to_feasibile_space_via_slsqp to nonlinear inequality constraints. Furthermore, it could make sense to change retry_on_optimization_warning from a bool to an int, so that more than one retry can be performed. What do you think?

Describe the solution you'd like to see implemented in BoTorch.

Regarding extending project_to_feasibile_space_via_slsqp:

• Simple idea: just pass through the callables of the nonlinear constraints and directly hook them into the slsqp solver of scipy. In case that there is only a subtle constraint violation of the nonlinear ones and the starting point is close to the feasible region, I would hope that the sqlsqp solver would be able to find the feasible region.
• More advanced idea: Linearize the nonlinear constraints via first order taylor expansions around the starting point, solve the problem for the linear constraints, check if nonlinear constraints are fulfilled, else iterate ...

Regarding retry_on_optimization_warning:

• I would change the signature to Union[bool|int] = 1, which would amount to one restart, True would be mapped to 1 and False to zero.
• And then I would introduce a while loop there instead of the single retry.

What is your opinion on this?

Describe any alternatives you've considered to the above solution.

No response

Is this related to an existing issue in BoTorch or another repository? If so please include links to those Issues here.

experimental-design/bofire#740

Pull Request

Yes

Code of Conduct

• I agree to follow BoTorch's Code of Conduct

[esantorella]

What is project_to_feasibile_space_via_slsqp? I'm not seeing any code search results for this.

[jduerholt]

What is project_to_feasibile_space_via_slsqp? I'm not seeing any code search results for this.

Here you go:

botorch/botorch/optim/parameter_constraints.py

Line 722 in c1aa7dd

def project_to_feasible_space_via_slsqp(

[esantorella]

Ah, thanks, just a tiny typo.

Supporting nonlinear inequality constraints in project_to_feasible_space_via_slsqp: Seems like a great idea! I'm not sure why they're not supported now, considering that gen_candidates_scipy does handle nonlinear_inequality_constraints and ultimately passes them to Scipy optimize, and project_to_feasible_space_via_slsqp also uses Scipy optimize. I'd be happy to accept a PR for that.

Using an int instead of bool for retry on optimization warning: I'm not opposed, since it wouldn't make the code any more complex, but I'd be curious to hear more about the use case for this. Have you encountered a case where optimization succeeds on the Nth try and not on the first two?

[jduerholt]

@sdaulton was implementing the projection code in #3010. @sdaulton: Was there a specific reason for not implementing it? @esantorella: What would you recommend for starting here: just rely on slsqp to handle the nonlinear ones directly to guide it into the feasible region (for starting points, close to the feasible region, I hope it works, but I can also run some tests), or do you want to have the linearization approach directly implemented?

Regarding the retry, I will discuss internally and come back to this.

[jduerholt]

CC: @MrzvskK

[MrzvskK]

Thank you @jduerholt, I will work on creating a PR for this.

[esantorella]

What would you recommend for starting here: just rely on slsqp to handle the nonlinear ones directly to guide it into the feasible region (for starting points, close to the feasible region, I hope it works, but I can also run some tests), or do you want to have the linearization approach directly implemented?

You have probably thought about this more than I have, but I would hope that it would work to take the same approach towards nonlinear inequality constraints as in gen_candidates_scipy, by formatting the constraints with make_scipy_nonlinear_inequality_constraints and then passing them to minimize. Maybe you or MrzvskK could check that this works on whatever internal use case you have motivating this change and then add some unit tests showing that it works? Then if it doesn't, a more complex approach like linearization would be needed.

[MrzvskK]

Opened PR with v1 approach from your comment: make_scipy_nonlinear_inequality_constraints + SLSQP in project_to_feasible_space_via_slsqp, plus tests and enabling repair in _optimize_acqf_batch.