Fix shape mismatch error in CatCMASampler for categorical problems

[Furkan-rgb]

Motivation

Problem:

When using the CatCMA sampler in Optuna with categorical parameters, a shape mismatch error occurs during the optimization process. This error prevents the sampler from functioning correctly, hindering the hyperparameter optimization workflow.

Error Details:

ValueError: operands could not be broadcast together with shapes (10,33) (6,11) This error arises from the attempt to perform element-wise operations on arrays with incompatible shapes within the CatCMA optimizer. Specifically, the categorical parameters are encoded as ragged arrays (arrays of varying lengths), leading to broadcasting issues.

Root Cause:
The categorical parameters are being one-hot encoded with differing lengths based on the number of choices per categorical variable. When these ragged arrays are converted to NumPy arrays with dtype=object, they lose their uniform shape, causing mismatches during optimizer computations.

Description of the changes

To resolve the shape mismatch, ensure that all categorical parameters are encoded into fixed-size, flat arrays. This can be achieved by concatenating all one-hot encoded vectors into a single flat array with a consistent length across all trials. Here's how to implement this:

Calculate Total Categories: Determine the total number of categories across all categorical variables to establish a fixed size for the concatenated array.

total_categories = sum(len(space.choices) for space in categorical_search_space.values()) Encode Categorical Parameters: Replace the ragged array encoding with a flat array approach, ensuring each c has the same shape.

for t in solution_trials[:popsize]:
    assert t.value is not None, "completed trials must have a value"
    # Convert numerical parameters
    x = trans.transform({k: t.params[k] for k in numerical_search_space.keys()})
    
    # Initialize a flat array for all categorical parameters
    c = np.zeros(total_categories)
    offset = 0
    for k in categorical_search_space.keys():
        choices = categorical_search_space[k].choices
        v = t.params.get(k)
        if v is not None:
            index = choices.index(v)
            c[offset + index] = 1
        offset += len(choices)
    
    y = t.value if study.direction == StudyDirection.MINIMIZE else -t.value
    solutions.append(((x, c), y))

Impact of the Change:

Consistency: Ensures that all categorical parameter arrays have a consistent shape, eliminating broadcasting issues.
Compatibility: Aligns with the CatCMA optimizer's expectations, allowing seamless integration of categorical parameters.

Robustness: Prevents runtime errors related to array shape mismatches, enhancing the reliability of the optimization process.

To illustrate:
If you have:
cat1 = ['A', 'B', 'C'] # 3 choices
cat2 = ['X', 'Y', 'Z', 'W'] # 4 choices

Original method would try to create:

[[1, 0, 0],       # shape (3,)
[0, 1, 0, 0]]     # shape (4,)

→ Results in shape mismatch

New method creates:

[[1, 0, 0, 0],    # padded to max_length
[0, 1, 0, 0]]     # uniform shape (2, 4)

→ Works correctly with the optimizer

Contributor Agreements

• I agree to the contributor agreements.

[Furkan-rgb]

@HideakiImamura could you please have a look?

[HideakiImamura]

Thanks for the PR. It seems that the linter CI fails. Could you apply the linters as suggested here?

[HideakiImamura]

LGTM.