Skip to content

Update healthcheck statuses (#5020)#5020

Closed
shrutipatel31 wants to merge 1 commit into
facebook:mainfrom
shrutipatel31:export-D96412166
Closed

Update healthcheck statuses (#5020)#5020
shrutipatel31 wants to merge 1 commit into
facebook:mainfrom
shrutipatel31:export-D96412166

Conversation

@shrutipatel31
Copy link
Copy Markdown
Contributor

@shrutipatel31 shrutipatel31 commented Mar 13, 2026

Summary:

Update healthcheck status levels to better reflect their meaning and actionability:

  1. Early Stopping Analysis:

    • ESS explicitly disabled: PASS → INFO (surface configuration choice)
    • ESS enabled, working correctly: PASS → INFO (informational, no action needed)
    • ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)
  2. Complexity Rating:

    • Advanced tier: WARNING → INFO (not blocking, informational)
    • Unsupported tier: FAIL → WARNING (concerning but not blocking)
  3. Can Generate Candidates:

    • can_generate=False (recent trial): WARNING → INFO (expected behavior)
  4. Baseline Improvement:

    • All objectives improved: PASS → INFO (valuable experiment takeaway)
    • Any objective not improved: WARNING → INFO (informational, not actionable)
  5. Should Generate Candidates:

    • should_generate=False : WARNING → INFO

Reviewed By: bernardbeckerman

Differential Revision: D96412166

@meta-cla meta-cla Bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Mar 13, 2026
@meta-codesync
Copy link
Copy Markdown

meta-codesync Bot commented Mar 13, 2026

@shrutipatel31 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D96412166.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.75%. Comparing base (d016302) to head (1c24f7b).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5020      +/-   ##
==========================================
- Coverage   96.75%   96.75%   -0.01%     
==========================================
  Files         606      606              
  Lines       66152    66152              
==========================================
- Hits        64007    64006       -1     
- Misses       2145     2146       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@meta-codesync meta-codesync Bot changed the title Change healthcheck statuses for clearer semantic meanings Change healthcheck statuses for clearer semantic meanings (#5020) Mar 13, 2026
shrutipatel31 added a commit to shrutipatel31/Ax that referenced this pull request Mar 13, 2026
)

Summary:

Update healthcheck status levels to better reflect their semantic meaning and actionability:

1. Early Stopping Analysis:
   - ESS explicitly disabled: PASS → INFO (surface configuration choice)
   - ESS enabled, working correctly: PASS → INFO (informational, no action needed)
   - ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)

2. Complexity Rating:
   - Advanced tier: WARNING → INFO (not blocking, informational)
   - Unsupported tier: FAIL → WARNING (concerning but not blocking)

3. Can Generate Candidates:
   - can_generate=False (recent trial): WARNING → INFO (expected behavior)

4. Baseline Improvement:
   - All objectives improved: PASS → INFO (valuable experiment takeaway)
   - Any objective not improved: WARNING → INFO (informational, not actionable)

Differential Revision: D96412166
@meta-codesync meta-codesync Bot changed the title Change healthcheck statuses for clearer semantic meanings (#5020) Update healthcheck statuses (#5020) Mar 16, 2026
shrutipatel31 added a commit to shrutipatel31/Ax that referenced this pull request Mar 16, 2026
Summary:

Update healthcheck status levels to better reflect their meaning and actionability:

1. Early Stopping Analysis:
   - ESS explicitly disabled: PASS → INFO (surface configuration choice)
   - ESS enabled, working correctly: PASS → INFO (informational, no action needed)
   - ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)

2. Complexity Rating:
   - Advanced tier: WARNING → INFO (not blocking, informational)
   - Unsupported tier: FAIL → WARNING (concerning but not blocking)

3. Can Generate Candidates:
   - can_generate=False (recent trial): WARNING → INFO (expected behavior)

4. Baseline Improvement:
   - All objectives improved: PASS → INFO (valuable experiment takeaway)
   - Any objective not improved: WARNING → INFO (informational, not actionable)

Differential Revision: D96412166
@shrutipatel31 shrutipatel31 force-pushed the export-D96412166 branch 2 times, most recently from 538999a to 314c6f3 Compare March 16, 2026 21:10
shrutipatel31 added a commit to shrutipatel31/Ax that referenced this pull request Mar 16, 2026
Summary:

Update healthcheck status levels to better reflect their meaning and actionability:

1. Early Stopping Analysis:
   - ESS explicitly disabled: PASS → INFO (surface configuration choice)
   - ESS enabled, working correctly: PASS → INFO (informational, no action needed)
   - ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)

2. Complexity Rating:
   - Advanced tier: WARNING → INFO (not blocking, informational)
   - Unsupported tier: FAIL → WARNING (concerning but not blocking)

3. Can Generate Candidates:
   - can_generate=False (recent trial): WARNING → INFO (expected behavior)

4. Baseline Improvement:
   - All objectives improved: PASS → INFO (valuable experiment takeaway)
   - Any objective not improved: WARNING → INFO (informational, not actionable)

Differential Revision: D96412166
shrutipatel31 added a commit to shrutipatel31/Ax that referenced this pull request Mar 17, 2026
Summary:

Update healthcheck status levels to better reflect their meaning and actionability:

1. Early Stopping Analysis:
   - ESS explicitly disabled: PASS → INFO (surface configuration choice)
   - ESS enabled, working correctly: PASS → INFO (informational, no action needed)
   - ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)

2. Complexity Rating:
   - Advanced tier: WARNING → INFO (not blocking, informational)
   - Unsupported tier: FAIL → WARNING (concerning but not blocking)

3. Can Generate Candidates:
   - can_generate=False (recent trial): WARNING → INFO (expected behavior)

4. Baseline Improvement:
   - All objectives improved: PASS → INFO (valuable experiment takeaway)
   - Any objective not improved: WARNING → INFO (informational, not actionable)

Reviewed By: bernardbeckerman

Differential Revision: D96412166
shrutipatel31 added a commit to shrutipatel31/Ax that referenced this pull request Mar 17, 2026
Summary:

Update healthcheck status levels to better reflect their meaning and actionability:

1. Early Stopping Analysis:
   - ESS explicitly disabled: PASS → INFO (surface configuration choice)
   - ESS enabled, working correctly: PASS → INFO (informational, no action needed)
   - ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)

2. Complexity Rating:
   - Advanced tier: WARNING → INFO (not blocking, informational)
   - Unsupported tier: FAIL → WARNING (concerning but not blocking)

3. Can Generate Candidates:
   - can_generate=False (recent trial): WARNING → INFO (expected behavior)

4. Baseline Improvement:
   - All objectives improved: PASS → INFO (valuable experiment takeaway)
   - Any objective not improved: WARNING → INFO (informational, not actionable)

Reviewed By: bernardbeckerman

Differential Revision: D96412166
shrutipatel31 added a commit to shrutipatel31/Ax that referenced this pull request Mar 17, 2026
Summary:

Update healthcheck status levels to better reflect their meaning and actionability:

1. Early Stopping Analysis:
   - ESS explicitly disabled: PASS → INFO (surface configuration choice)
   - ESS enabled, working correctly: PASS → INFO (informational, no action needed)
   - ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)

2. Complexity Rating:
   - Advanced tier: WARNING → INFO (not blocking, informational)
   - Unsupported tier: FAIL → WARNING (concerning but not blocking)

3. Can Generate Candidates:
   - can_generate=False (recent trial): WARNING → INFO (expected behavior)

4. Baseline Improvement:
   - All objectives improved: PASS → INFO (valuable experiment takeaway)
   - Any objective not improved: WARNING → INFO (informational, not actionable)

5. Should Generate Candidates:
   - should_generate=False : WARNING → INFO

Reviewed By: bernardbeckerman

Differential Revision: D96412166
Summary:

Update healthcheck status levels to better reflect their meaning and actionability:

1. Early Stopping Analysis:
   - ESS explicitly disabled: PASS → INFO (surface configuration choice)
   - ESS enabled, working correctly: PASS → INFO (informational, no action needed)
   - ESS not enabled but could save ≥10%: WARNING → INFO (nudge for feature upsell; not blocking the experiment)

2. Complexity Rating:
   - Advanced tier: WARNING → INFO (not blocking, informational)
   - Unsupported tier: FAIL → WARNING (concerning but not blocking)

3. Can Generate Candidates:
   - can_generate=False (recent trial): WARNING → INFO (expected behavior)

4. Baseline Improvement:
   - All objectives improved: PASS → INFO (valuable experiment takeaway)
   - Any objective not improved: WARNING → INFO (informational, not actionable)

5. Should Generate Candidates:
   - should_generate=False : WARNING → INFO

Reviewed By: bernardbeckerman

Differential Revision: D96412166
@meta-codesync
Copy link
Copy Markdown

meta-codesync Bot commented Mar 17, 2026

This pull request has been merged in 71b6c1a.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported Merged meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants