v0.9.0b4

[c-bata]

Human-in-the-loop Optimization

$ pip install Pillow optuna==3.1.0 optuna-dashboard==0.9.0b4

from __future__ import annotations

import os
import textwrap
import threading
import time
from typing import NoReturn
from wsgiref.simple_server import make_server

import optuna
from PIL import Image
from optuna.trial import TrialState

from optuna_dashboard import ObjectiveChoiceWidget, save_note, ObjectiveSliderWidget
from optuna_dashboard import register_objective_form_widgets
from optuna_dashboard import set_objective_names
from optuna_dashboard import wsgi
from optuna_dashboard.artifact import upload_artifact
from optuna_dashboard.artifact.file_system import FileSystemBackend


storage = optuna.storages.InMemoryStorage()
base_path = os.path.join(os.path.dirname(__file__), "artifact")
artifact_backend = FileSystemBackend(base_path=base_path)


def suggest_and_generate_image(study: optuna.Study) -> None:
    # Ask new parameters
    trial = study.ask()
    r = trial.suggest_int("r", 0, 255)
    g = trial.suggest_int("g", 0, 255)
    b = trial.suggest_int("b", 0, 255)

    # Generate image
    image_path = f"tmp/sample-{trial.number}.png"
    image = Image.new("RGB", (320, 240), color=(r,g,b))
    image.save(image_path)

    # Upload Artifact
    artifact_id = upload_artifact(artifact_backend, trial, image_path)

    # Save Note
    note = textwrap.dedent(f'''\
    ## Trial {trial.number}
    
    ![generated-image](/artifacts/{study._study_id}/{trial._trial_id}/{artifact_id})
    ''')
    save_note(trial, note)


def start_preferential_optimization() -> NoReturn:
    # Create Study
    study = optuna.create_study(
        study_name="Human-in-the-loop Optimization",
        storage=storage,
        directions=["minimize", "maximize"]
    )
    set_objective_names(study, ["Human Perception Score", "Looks Yellow?"])
    register_objective_form_widgets(study, widgets=[
        ObjectiveChoiceWidget(
            choices=["Good 👍", "Bad 👎"],
            values=[-1, 1],
            description="Please input your score!",
        ),
        ObjectiveSliderWidget(
            min=1,
            max=10,
            step=1,
            description="Higher is better.",
        ),
    ])

    # Start Human-in-the-loop Optimization
    n_batch = 3
    while True:
        running_trials = study.get_trials(deepcopy=False, states=(TrialState.RUNNING,))
        if len(running_trials) >= n_batch:
            time.sleep(5)
            continue
        suggest_and_generate_image(study)


def main() -> None:
    if not os.path.exists(base_path):
        os.mkdir(base_path)

    # Start Dashboard server on background
    app = wsgi(storage, artifact_backend=artifact_backend)
    httpd = make_server("127.0.0.1", 8080, app)
    thread = threading.Thread(target=httpd.serve_forever)
    thread.start()

    # Run optimize loop
    try:
        start_preferential_optimization()
    except KeyboardInterrupt:
        httpd.shutdown()
        httpd.server_close()
        thread.join()


if __name__ == "__main__":
    main()

What's Changed

• Fix bug of custom user widgets and enhance artifact support. by @c-bata in Fix bug of custom user widgets and enhance artifact support. #375
• Update best trials when state is updated by @c-bata in Update best trials when state is updated #376
• Bump the version up to v0.9.0b4 by @c-bata in Bump the version up to v0.9.0b4 #377

Full Changelog: v0.9.0b3...v0.9.0b4

---

This discussion was created from the release v0.9.0b4.

[hrntsm]

Hi,

Human-in-the-loop is a very interesting feature.
Looking at the sample code above, it looks like it registers the objective with a register_objective_form_widgets, so it is not possible to register the result in the objective without using widgets.
Can it be combined with a simple function return value?

[knshnb]

Hi, thanks for your interest in the human-in-the-loop feature!
Sorry for not documenting this yet, but you can specify values to tell from trial.user_attrs by using ObjectiveUserAttrRef (

optuna-dashboard/optuna_dashboard/_objective_form_widget.py

Lines 99 to 107 in 957a8fd

	@dataclass
	class ObjectiveUserAttrRef:
	key: str
	def to_dict(self) -> UserAttrRefJSON:
	return {
	"type": "user_attr",
	"key": self.key,
	}

).

Also, we are going to support another interface to register human-evaluated scores to trial.user_attrs instead of directly telling them: #411. With this, you can write any Python script to compute an objective score from them.

[hrntsm]

Hi @knshnb, Thanks for the reply!
I was able to use the ObjectiveUserAttrRef you taught me to get an objective value for the function.
I've looked at another method and it looks useful too. Looking forward to seeing it implemented.