Skip to content

generate_report fails when less than three optimizers due to Friedman testing #223

@andreas122001

Description

@andreas122001

The script carps.analysis.generate_report fails on critical difference plotting if less than three optimizers are given since Friedman test requires multiple samples.
E.g. error:

ValueError: At least 3 sets of samples must be given for Friedman test, got 2.

Todo: change the generate_report function to only do reports depending on Friedman tests if there are more than two optimizers being compared.

Could be something like wrapping it in an if-statement

n_optimizers = len(df['optimizer_id'].value_counts())
if n_optimizers >= 3:
    # critical difference
    # ranks over time
    # etc.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions