Add TuRBOSampler

package/samplers/turbo/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Preferred Networks, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/samplers/turbo/README.md

@@ -0,0 +1,71 @@
+---
+author: Optuna Team
+title: TuRBOSampler
+description: This sampler performs Bayesian optimization in adaptive trust regions using Gaussian Processes
+tags: [sampler, Bayesian optimization]
+optuna_versions: [4.6.0]
+license: MIT License
+---
+
+## Abstract
+
+TuRBOSampler implements Bayesian optimization with trust regions. It places local trust regions around the current best solutions and fits Gaussian Process (GP) models within those regions. Operating within adaptive local regions reduces high-dimensional sample complexity, yielding accurate fits with fewer trials.
+
+Please refer to the paper, [Scalable Global Optimization via Local Bayesian Optimization](https://proceedings.neurips.cc/paper_files/paper/2019/file/6c990b7aca7bc7058f5e98ea909e924b-Paper.pdf) for more information.
+
+## APIs
+
+- `TuRBOSampler(*, n_startup_trials: int = 4, n_trust_region: int = 5, success_tolerance: int = 3, failure_tolerance: int = 5, seed: int | None = None, independent_sampler: BaseSampler | None = None, deterministic_objective: bool = False, warn_independent_sampling: bool = True)`
+  - `n_startup_trials`: Number of initial trials PER TRUST REGION. Default is 2. As suggested in the original paper, consider setting this to 2\*(number of parameters).
+  - `n_trust_region`: Number of trust regions. Default is 5.
+  - `success_tolerance`: Number of consecutive successful iterations required to expand the trust region. Default is 3.
+  - `failure_tolerance`: Number of consecutive failed iterations required to shrink the trust region. Default is 5. As suggested in the original paper, consider setting this to max(5, number of parameters).
+  - `init_length`: The initial size of the trust region. Defaults to 0.8.
+  - `max_length`: Maximum size of the trust region. If the trust region grows beyond this value, its size is clipped to this value. Defaults to 1.6.
+  - `min_length`: Minimum size of the trust region. If the trust region shrinks below this value, the region is regarded as fully explored and as having reached a local optimum. In this case, the current trust region is discarded and a new one is initialized with size `init_length`. Defaults to 0.5\*\*7.
+  - `seed`: Random seed to initialize internal random number generator. Defaults to :obj:`None` (a seed is picked randomly).
+  - `independent_sampler`: Sampler used for initial sampling (for the first `n_startup_trials` trials) and for conditional parameters. Defaults to :obj:`None` (a random sampler with the same `seed` is used).
+  - `deterministic_objective`: Whether the objective function is deterministic or not. If :obj:`True`, the sampler will fix the noise variance of the surrogate model to the minimum value (slightly above 0 to ensure numerical stability). Defaults to :obj:`False`. Currently, all the objectives will be assume to be deterministic if :obj:`True`.
+  - `warn_independent_sampling`: If this is :obj:`True`, a warning message is emitted when the value of a parameter is sampled by using an independent sampler, meaning that no GP model is used in the sampling. Note that the parameters of the first trial in a study are always sampled via an independent sampler, so no warning messages are emitted in this case.
+
+Note that categorical parameters are currently unsupported, and multi-objective optimization is not available.
+
+## Installation
+
+```shell
+$ pip install torch scipy
+```
+
+## Example
+
+```python
+import optuna
+import optunahub
+
+
+def objective(trial: optuna.Trial) -> float:
+    x = trial.suggest_float("x", -5, 5)
+    y = trial.suggest_float("y", -5, 5)
+    return x**2 + y**2
+
+
+sampler = optunahub.load_module(package="samplers/turbo").TuRBOSampler()
+study = optuna.create_study(sampler=sampler)
+study.optimize(objective, n_trials=200)
+
+```
+
+## Others
+
+### Bibtex
+
+```
+@inproceedings{eriksson2019scalable,
+  title = {Scalable Global Optimization via Local {Bayesian} Optimization},
+  author = {Eriksson, David and Pearce, Michael and Gardner, Jacob and Turner, Ryan D and Poloczek, Matthias},
+  booktitle = {Advances in Neural Information Processing Systems},
+  pages = {5496--5507},
+  year = {2019},
+  url = {http://papers.nips.cc/paper/8788-scalable-global-optimization-via-local-bayesian-optimization.pdf},
+}
+```

package/samplers/turbo/__init__.py

@@ -0,0 +1,4 @@
+from .sampler import TuRBOSampler
+
+
+__all__ = ["TuRBOSampler"]