Skip to content

Conversation

@fusawa-yugo
Copy link
Contributor

Motivation

Guides on benchmarks in OptunaHub would be useful for users.

Description of the changes

I added use cases for normal problem, constrained problem, and multi-objective problem.

@fusawa-yugo fusawa-yugo marked this pull request as draft September 3, 2025 07:52
@codecov
Copy link

codecov bot commented Sep 3, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.37%. Comparing base (ff4ea79) to head (0dac627).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #110   +/-   ##
=======================================
  Coverage   81.37%   81.37%           
=======================================
  Files           9        9           
  Lines         204      204           
=======================================
  Hits          166      166           
  Misses         38       38           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fusawa-yugo fusawa-yugo force-pushed the fusawa-yugo/benchmark-tutorial branch from 2ef8525 to bd4e292 Compare September 5, 2025 05:18
@fusawa-yugo fusawa-yugo force-pushed the fusawa-yugo/benchmark-tutorial branch from bd4e292 to 7a45bc5 Compare September 5, 2025 08:51
@fusawa-yugo fusawa-yugo marked this pull request as ready for review September 5, 2025 08:52
Copy link
Member

@y0z y0z left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I leave some comments.

How to Use Benchmarks
=====================

In this tutorial, we will explain how to use benchmarks in optunahub.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In this tutorial, we will explain how to use benchmarks in optunahub.
In this tutorial, we will explain how to use benchmarks in OptunaHub.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:alt: Optimization History
:align: center
:width: 800px

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add an example to use OptunaHub Benchmarks with optimizers other than Optuna, like https://medium.com/optuna/optunahub-benchmarks-a-new-feature-to-use-register-various-benchmark-problems-694401524ce0.
image

Copy link
Member

@y0z y0z Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc of the evaluate method is here.

:alt: Optimization History
:align: center
:width: 800px

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add an explanation of evaluate_constraints, which allows constrained optimization with optimizers other than Optuna by directly evaluating constraints with user-provided parameters.

Copy link
Member

@y0z y0z left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@y0z y0z merged commit 7f4f15c into optuna:main Sep 26, 2025
13 checks passed
@nabenabe0928 nabenabe0928 added the document Documentation related. label Oct 8, 2025
@nabenabe0928 nabenabe0928 added this to the v0.4.0 milestone Oct 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

document Documentation related.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants