Version 2.0.0a1

Big Changes

• We redesigned the scenario class completely. The scenario is implemented as a dataclass now and holds only environment variables (like limitations or save directory). Everything else was moved to the components directly.
• We removed runtime optimization completely (no adaptive capping or imputing anymore).
• We removed the command-line interface and restructured everything alongside. Since SMAC was building upon the command-line interface (especially in combination with the scenario), it was complicated to understand the behavior or find specific implementations. With the removal, we re-wrote everything in python and re-implemented the feature of using scripts as target functions.
• Introducing trials: Each config/seed/budget/instance calculation is a trial.
• The configuration chooser is integrated into the SMBO object now. Therefore, SMBO finally implements an ask-tell interface now.
• Facades are redesigned so that they accept instantiated components directly. If a component is not passed, a default component is used, which is specified for each facade individually in the form of static methods. You can use those static methods directly to adapt a component to your choice.
• A lot of API changes and renamings (e.g., RandomConfigurationChooser -> RandomDesign, Runhistory2EPM -> RunHistoryEncoder).
• Ambiguous variables are renamed and unified across files.
• Dependencies of modules are reduced drastically.
• We incorporated Pynisher 1.0, which ensures limitations cross-platform.
• We incorporated ConfigSpace 0.6, which simplified our examples.
• Examples and documentation are completely reworked. Examples use the new ConfigSpace, and the documentation is adapted to version 2.0.
• Transparent target function signatures: SMAC checks now explicitly if an argument is available (the required arguments are now specified in the intensifier). If there are more arguments that are not passed by SMAC, a warning is raised.
• Components implement a meta property now, all of which describe the initial state of SMAC. The facade collects all metadata and saves the initial state of the scenario.
• Improved multi-objective in general: RunHistory (in addition to RunHistoryEncoder) both incorporates the multi-objective algorithm. In other words, if the multi-objective algorithm changes the output, it directly affects the optimization process.
• Configspace is saved in json only
• StatusType is saved as integer and not as dict anymore
• We changed the behavior of continuing a run:
  • SMAC automatically checks if a scenario was saved earlier. If there exists a scenario and the initial state is the same, SMAC automatically loads the previous data. However, continuing from that run is not possible yet.
  • If there was a scenario earlier, but the initial state is different, then the user is asked to overwrite the run or to still continue the run although the state is different (Note that this only can happen if the name specified in the scenario is the same). Alternatively, an old to the old run is added (e.g., the name was test, it becomes test-old).
  • The initial state of the SMAC run also specifies the name (if no name in the scenario is specified). If the user changes something in the code base or in the scenario, the name and, therefore, the save location automatically changes.

New Features

• Added a new termination feature: Use terminate_cost_threshold in the scenario to stop the optimization after a configuration was evaluated with a cost lower than the threshold.
• Callbacks are completely redesigned. Added callbacks to the facade are called in different positions in the Bayesian optimization loop.
• The multi-objective algorithm MeanAggregationStrategy supports objective weights now.
• RunHistory got more methods like get_incumbent or get_pareto_front.

Fixes

• You ever noticed that the third configuration has no origin? It's fixed now.
• We fixed ParEGO (it updates every time training is performed now).

Optimization Changes

• Changed initial design behavior
  • You can add additional configurations now.
  • max_ratio will limit both n_configs and n_configs_per_hyperparameter but not additional configurations
  • Reduced default max_ratio to 0.1.

Code Related

• Converted all unittests to pytests.
• Instances, seeds, and budgets can be set to none now. However, mixing none and non-none will throw an exception.

Version 1.4.0

Features

• BOinG: A two-stage Bayesian optimization approach to allow the
  optimizer to focus on the most promising regions.
• TurBO: Reimplementaion of TurBO-1 algorithm.
• Updated pSMAC: Can pass arbitrary SMAC facades now. Added example and fixed tests.

Improvements

• Enabled caching for multi-objectives (#872). Costs are now normalized in get_cost
  or optionally in average_cost/sum_cost/min_cost to receive a single float value. Therefore,
  the cached cost values do not need to be updated everytime a new entry to the runhistory was added.

Interface changes

• We changed the location of Gaussian processes and random forests. They are in the folders
  epm/gaussian_process and epm/random_forest now.
• Also, we restructured the optimizer folder and therefore the location of the acquisition functions
  and configuration chooser.
• Multi-objective functions are located in the folder multi_objective.
• pSMAC facade was moved to the facade directory.

Version 1.3.4

• Added reference to JMLR paper.
• Typos in documentations.
• Code more readable since all typings are imported at the beginning of the file.
• Updated stale bot options.

Version 1.3.3

• Hotfix: Since multi-objective implementation depends on normalized costs, it now is ensured that the
  cached costs are updated everytime a new entry is added.
• Removed mac-specific files.
• Added entry point for cli.
• Added ConfigSpace to third known parties s.t. sorting should be the same across different
  operating systems.
• Fixed bugs in makefile in which tools were specified incorrectly.
• Executed isort/black on examples and tests.
• Updated README.
• Fixed a problem, which incremented time twice before taking log (#833).
• New wrapper for multi-objective models (base_uncorrelated_mo_model). Makes it easier for
  developing new multi-objective models.
• Raise error if acquisition function is incompatible with the epm models.
• Restricting pynisher.

Version 1.3.2

• Added stale bot support.
• If package version 0.0.0 via get_distribution is found, the version of the module is used
  instead.
• Removed tox.ini.
• Moved requirements.txt to setup.py.
• Added multi-objective support for ROAR.
• Added notes in documentation that SMAC4MF is the closest implementation to BOHB/HpBandSter.

Version 1.3.1

• Added Python 3.7 support again.

Version 1.3

Features

• PiBO: Augment the acquisition function by multiplying by a pdf given by the user.
  The prior then decays over time, allowing for the optimization to carry on as per default.
• The RunHistory can now act as a Mapping in that you can use the usual methods you can use on dicts, i.e. len(rh), rh.items(), rh[key]. Previously this was usually done by accessing rh.data which is still possible.

Minor Changes

• Updated the signature of the ROAR facade to match with it's parent class SMAC4AC. Anyone relying on the output directory without specifying an explicit run_id to a ROAR facade should now expect to see the output directory at run_0 instead of run_1. See #827.

Code-Quality

• Updated and integrated flake8, mypy, black, and isort.

Documentation

• SMAC uses automl_sphinx_theme now and therefore the API is displayed nicer.

Version 1.2

Features

• Added multi-objective optimization via Mean-Aggregation or Par-EGO (#817, #818). Both approaches normalize
  the costs objective-wise based on all data in the history.

Major Changes

• Results are instantly saved by default now. That means, runhistory.json is saved every time
  a trial is added.
• Determinstic behaviour (defined in scenario) is default now. Calling a function/TAE with the same
  seed and configuration is expected to be the same.
• Limit resources behaviour is by default false now. This is particually important because pynisher
  does not work on all machines (e.g. Colab, Mac, Windows, ...) properly.
• Renamed scenario object save_results_instantly to save_instantly.
• Added multi_objectives as scenario argument.
• Expanded cost_for_crash for multi-objective support.

Examples

• Integrated spear_qcp example for commandline.
• Python examples are now executed so that the output in the documentation is shown.
• Added multi-objective example.

Documentation

• Added runhistory page.

Workflow Clean-up

• Adds PEP 561 compliance (exports types so other packages can be aware of them).
• Allow manual workflow_dispatch on actions that might require it (can manually trigger them from github UI).
• Prevent the double trigger of actions by making push and pull_request and more strict.
• A push to a pull request should no longer cause double the amount of tests to run (along with the other workflows that had on: [push, pull_request].
• Some general cleanup, giving names to some actions, adding some linebreaks to break up things, ...
• Command-line examples are tested again.
• pytest.yaml:
  • Now scheduled to auto run everyday instead of every week.
  • Clean up the body of the steps and move some things to env var.
  • Scaffold for matrix that includes windows and mac testing (currently excluded, see comments).
  • Includes tests for Python 3.10.
  • Changed the boolean flags in the matrix to just be a categorical, easier to read.

Minor Changes

• Specified that dask should not cache functions/results (#803) .
• Handles invalid configuration vectors gracefully (#776).
• Specified scenario docs that also SMAC options can be used.
• Docs display init methods now.
• Parameters in the docs are shown first now.
• Successive Halving only warns you once if one worker is used only.
• Statistics are better readable now.
• Sobol sequence does not print warnings anymore.

Release 1.1.1

Minor Changes

• Added comparison between SMAC and similar tools.
• Updated installation guide.
• Added a warning that CLI is only available when installing from GitHub.

Release 1.1.0

Major Changes

• Option to use an own stopping strategy using IncorporateRunResultCallback.
• Documentation was updated thoroughly. A new theme with a new structure is provided and all pages
  have been updated. Also, the examples revised and up-to-date.
• Changed scripts/smac to scripts/smac.py.

Minor Changes

• README.md updated.
• CITATION.cff added.
• Made smac-validate.py consistent with runhistory and tae. (#762)
• minR, maxR and use_ta_time can now be initialized by the scenario. (#775)
• ConfigSpace.util.get_one_exchange_neighborhood's invalid configurations are ignored. (#773)

Bug Fixes

• Fixed an incorrect adaptive capping behaviour. (#749)
• Avoid the potential ValueError raised by LocalSearch._do_search. (#773)