Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces

✨️ Accepted to the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining ✨
(KDD 2026)

Kaito Baba   Yoshihiko Ozaki   Shuhei Watanabe

We propose conditional PED-ANOVA (condPED-ANOVA), a principled framework for estimating hyperparameter importance (HPI) in conditional search spaces, where the presence or domain of a hyperparameter can depend on other hyperparameters. Although the original PED-ANOVA provides a fast and efficient way to estimate HPI within the top-performing regions of the search space, it assumes a fixed, unconditional search space and therefore cannot properly handle conditional hyperparameters. To address this, we introduce a conditional HPI for top-performing regions and derive a closed-form estimator that accurately reflects conditional activation and domain changes. Experiments show that naive adaptations of existing HPI estimators yield misleading or uninterpretable importances in conditional settings, whereas condPED-ANOVA consistently provides meaningful importances that reflect the underlying conditional structure.

　

🚀 Quick Start 🚀

cond_ped_anova is fully compatible with Optuna’s built-in hyperparameter importance API.

1. Installation

uv add git+https://github.com/kAIto47802/condPED-ANOVA.git
# If you're using pip:
# pip install git+https://github.com/kAIto47802/condPED-ANOVA.git

2. Usage

import optuna
from cond_ped_anova import CondPedAnovaImportanceEvaluator


def objective(trial: optuna.trial.Trial) -> float:
    c = trial.suggest_float("c", 0.0, 1.0)
    if c < 0.5:
        x = trial.suggest_float("x", -5.0, -2.0)
        return x
    else:
        y = trial.suggest_float("y", 2.0, 5.0)
        return y


sampler = optuna.samplers.RandomSampler(seed=42)
study = optuna.create_study(direction="minimize", sampler=sampler)
study.optimize(objective, n_trials=1000)

evaluator = CondPedAnovaImportanceEvaluator()
importance = optuna.importance.get_param_importances(study, evaluator=evaluator)

📊 Reproducing the Results 📊

All experiments in our paper are fully reproducible.

1. Clone this Repository & Install Dependencies

git clone https://github.com/kAIto47802/condPED-ANOVA.git

uv sync --python 3.13 --extra experiments
# If you're using pip:
# pip install --upgrade pip  # enable PEP 660 support
# pip install -e '.[experiments]'

2. Run Experiments

Running the commands below reproduces the corresponding experiments and regenerates the exact figures:

• condPED-ANOVA on synthetic problems (Figures 1, 7, 11, and 12):

  ./run_cond_ped_anova.sh

• Baseline methods on synthetic problems (Figures 2, 9, and 10):

  ./run_baselines.sh

• Ablation study (Figures 3 and 4):

  ./run_ablations.sh

• condPED-ANOVA and baseline methods on real-world benchmarks (Figures 5 and 15 and Tables 1, 2, 3, and 4):

  # prepare YAHPO Gym data
  git clone https://github.com/slds-lmu/yahpo_data.git
  
  ./run_yahpo_gym.sh

• Runtime comparison (Figure 6):

  ./run_runtime_comparison.sh

• condPED-ANOVA results with different $N$ (Figure 8):

  ./run_cond_ped_anova_different_n.sh

• Additional experiments on synthetic problems (Figures 13 and 14):

  ./run_cond_ped_anova_additional.sh

3. Check the Results

The generated figures will be saved in the figures/ directory. The raw experiment outputs (pickled results) and the values used in the tables are saved under results/.

Note

If a LaTeX environment is not available, the figures will be rendered without LaTeX, which may slightly change font rendering and layout.

🔖 Citation 🔖

If you find condPED-ANOVA useful in your research, please consider citing the following paper:

@inproceedings{baba2026condpedanova,
  title     = {Conditional {PED-ANOVA}: Hyperparameter Importance in Hierarchical \& Dynamic Search Spaces},
  author    = {Baba, Kaito and Ozaki, Yoshihiko and Watanabe, Shuhei},
  year      = {2026},
  month     = {August},
  booktitle = {Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2},
  publisher = {Association for Computing Machinery},
  address   = {Jeju Island, Republic of Korea},
  doi       = {10.1145/3770855.3817758},
  isbn      = {979-8-4007-2259-2/2026/08},
}