Skip to content

Commit 47c7bc3

Browse files
shrutipatel31meta-codesync[bot]
authored andcommitted
Improve EarlyStoppingAnalysis healthcheck messages (#5080)
Summary: Pull Request resolved: #5080 Two message improvements in EarlyStoppingAnalysis Healthcheck : 1. ESS enabled but strategy missing for MOO/constrained: Clarify that this happens when loading a historical experiment without passing the original strategy to EarlyStoppingAnalysis. 2. No MapMetrics case: Prefix with 'Early stopping strategy is enabled, but...' and add action steps to fix the metric configuration. Reviewed By: bernardbeckerman Differential Revision: D97173502 fbshipit-source-id: d4a3e189a28733ec3159bde338c25699f7e050e6
1 parent ca1d502 commit 47c7bc3

1 file changed

Lines changed: 14 additions & 9 deletions

File tree

ax/analysis/healthcheck/early_stopping_healthcheck.py

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -272,12 +272,13 @@ def _report_early_stopping_status(
272272
)
273273
return self._create_card(
274274
subtitle=(
275-
f"This experiment has {n_stopped} early stopped trial(s), but no "
276-
"early stopping strategy was provided. For multi-objective or "
277-
"constrained experiments, please pass the original "
278-
"early_stopping_strategy used during the experiment to "
279-
"EarlyStoppingAnalysis for accurate reporting of metrics and "
280-
"savings."
275+
f"This experiment has {n_stopped} early stopped trial(s), but "
276+
"no early stopping strategy was provided to "
277+
"EarlyStoppingAnalysis. This typically happens when loading a "
278+
"historical experiment where early stopping was used, but the "
279+
"original strategy object was not passed to this analysis. "
280+
"To get accurate savings reporting, pass the original "
281+
"early_stopping_strategy to EarlyStoppingAnalysis."
281282
),
282283
status=HealthcheckStatus.WARNING,
283284
df=df,
@@ -340,9 +341,13 @@ def _report_early_stopping_status(
340341
)
341342
return self._create_card(
342343
subtitle=(
343-
f"None of the {len(target_ess_metric_names)} metrics used for "
344-
f"early stopping are MapMetrics with map data. Early stopping "
345-
f"requires time-series data."
344+
f"Early stopping strategy is enabled, but none of the "
345+
f"{len(target_ess_metric_names)} metrics used for early "
346+
f"stopping are MapMetrics with map data. Early stopping "
347+
f"requires time-series (map) data to make stopping decisions. "
348+
f"To fix this, ensure that the metrics used by your early "
349+
f"stopping strategy are MapMetrics and that map data is "
350+
f"attached to the experiment."
346351
),
347352
status=HealthcheckStatus.FAIL,
348353
df=df,

0 commit comments

Comments
 (0)