Skip to content

cate_scoring default learner is oversized for its own per-fold, per-arm call pattern #1037

Description

@jeongyoonlee

Low priority.

The default LGBMRegressor(num_leaves=64, ...) merged in #1030 (cate_scoring.py:63, :275, :399) is inherited from visualize.py's TMLE helpers (:343, :453, :696, :740), which fit on the full frame. dr_score / plug_in_t_score fit per-fold and per-arm — with the default n_folds=5 that is 10 fits on roughly n/10 rows each, where 64 leaves is well past what the slice supports. #1030 added verbose=-1, which silences the resulting "No further splits with positive gain" chatter without addressing the cause.

Two possible improvements, neither urgent:

  1. A smaller num_leaves for this call pattern, or one sized from the data.
  2. A docstring nudge on dr_score: comparing several candidate CATE models without an explicit learner refits 10 LGBM models per call, which is what compute_dr_pseudo_outcomes's own docstring advises avoiding by precomputing pseudo-outcomes once and passing pseudo_outcome_col.

Measured cost when #1030's three learner-less tests were added: tests/test_cate_scoring.py went from 17s to 37–55s.

Observed on master (477cd0a).

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions