Skip to content

bench: fix convex_fraction=0.5 as the default for mixed (stop searching it) #123

Description

@davorrunje

Context

The base-result ablation (PR #111, docs/benchmarks/alternate-base-result.md) compared mixed with a searched convex_fraction against mixed-fixed (the same construction with convex_fraction pinned at 0.5). Fixing it at 0.5 won 3/5 datasets — substantially on compas (+0.017 acc) and auto (−0.33 MSE) — and searching it complicates initialization (any fraction ≠ 0.5 triggers a gain/bias fixed-point in the init). Despite this evidence, the search still tunes convex_fraction by default: benchmarks/_common/search_spaces.py:35 has search_convex_fraction: bool = True.

Goal

Make convex_fraction = 0.5 (no search) the default for mixed, so runs get the better-and-simpler behavior without the --fix-convex-fraction opt-in.

Where

  • benchmarks/_common/search_spaces.py:suggest_config — flip the search_convex_fraction default to False (line ~35), and adjust the dependent branch (line ~98, if mode == "mixed" and search_convex_fraction).
  • benchmarks/_common/search.py:search / run_dataset — the pass-through default.
  • benchmarks/search.py (CLI) — the --fix-convex-fraction flag becomes the default; consider replacing it with --search-convex-fraction to opt back in.

Acceptance criteria

  • Default mixed runs fix convex_fraction = 0.5 (no search); opting into the search is explicit.
  • Existing tests updated (e.g. tests/benchmarks/test_search.py, test_search_cli.py).
  • The behavior change is noted in the results/docs.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestfollow-upDeferred work captured as a self-contained issue

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions