Commit ef1f1de
Add template method pattern to DerivedMetric base class (#4965)
Summary:
Pull Request resolved: #4965
Refactors `DerivedMetric` to implement a template-method `fetch_trial_data` that
consolidates shared data-fetching, validation, and optional relativization logic:
1. `_resolve_source_trial_indices(trial)` — subclass hook for cross-trial data
lookup; default returns `None` (use current trial).
2. `_lookup_base_data(trial, source_trial_indices)` — looks up cached metric data.
3. `_validate_input_metrics_present(df)` — checks all input metrics exist globally.
4. `_collect_arm_data(trial, df, source_trial_indices)` — collects per-arm input
metric DataFrames with per-arm missing-metric error handling.
5. `_relativize_arm_data(trial, arm_data, cached_df)` — when `relativize_inputs=True`,
relativizes each arm independently against the status quo from that arm's own
source trial using `Data.relativize` (delta method). This correctly handles
non-stationarity where the same SQ arm may have different metric values across
trials. The status quo arm is excluded from the output.
6. `_compute_derived_values(trial, arm_data)` — abstract subclass hook for
the actual computation.
The `relativize_inputs` parameter (default `False`) enables automatic
status-quo normalization of input metric values before the subclass
computation step. The `as_percent` parameter (default `True`) controls
whether relativized values are expressed as percentage change (50.0 for
a 50% improvement) or fractional change (0.5).
Reviewed By: bletham
Differential Revision: D94844067
fbshipit-source-id: cbed01770b02553342587b49bb466628ac3664dc1 parent 3a77c2b commit ef1f1de
2 files changed
Lines changed: 776 additions & 101 deletions
0 commit comments