Push logs_url to status API eagerly and via final PUT#54
Merged
ishandhanani merged 3 commits intomainfrom Apr 21, 2026
Merged
Conversation
Previously, when a recipe set `reporting.status.endpoint` but the cluster's srtslurm.yaml did not, `_report_metrics` in PostProcessStageMixin would silently no-op because it only read `load_cluster_config()`. The final `report_completed` PUT also fired before `run_postprocess`, so the "completed" status hit the collector before any rollup was generated. Net effect: the collector recorded lifecycle correctly but `logs_url` stayed null, and consumers (e.g. IBAR) could not find the S3 artifacts. Changes: - `StatusReporter.report_completed(exit_code, logs_url=None)` — final PUT now carries the artifact pointer when available. - New `StatusReporter.report_artifacts(logs_url)` — eager mid-run PUT that updates `logs_url` without flipping the lifecycle to `completed`. Fires as soon as S3 sync returns, before AI analysis runs, so a hanging or crashing analyzer cannot strand the pointer. - `run_postprocess(exit_code, reporter=None)` — accepts the reporter, pushes `logs_url` eagerly, and stashes it on `self._last_logs_url` for the caller's final `report_completed`. - Removed `_report_metrics`: duplicate PUT path that only read cluster config, missed recipe-defined endpoints, and is now fully covered by the reporter-based path that resolves both sources via `from_config`. - `do_sweep.run()` finally block reordered: postprocess runs before `report_completed`, and the final PUT reasserts `logs_url` idempotently. Benchmark results themselves are intentionally NOT forwarded through the status API. S3 remains the source of truth for artifacts; the collector only stores the pointer. Also bundles a small helper refactor in `core/config.py` (extract `find_cluster_config_path`, add `get_cluster_aliases`) that was pending in the working tree. Tests: - `test_status.py`: coverage for the new `logs_url` kwarg on `report_completed` and three cases for `report_artifacts` (disabled, empty URL, success without flipping status to completed). - `test_postprocess.py`: replaced `_report_metrics` assertions with the new shape — eager reporter push, stash on self, skip paths for no S3 URL and no reporter. All 589 tests pass. Ruff clean on touched files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four references in comments/docstrings/CLI help were naming a specific
external harness. Replaced with provider-neutral language ("upstream
harnesses", "downstream consumers"). No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
reporting.status.endpoints were silently skipped by the post-process dashboard PUT. The path only readload_cluster_config(), so any endpoint set in the recipe (rather than cluster srtslurm.yaml) never receivedlogs_url. Consequence: the collector recorded job lifecycle correctly butlogs_urlstayednull, and consumers could not find the S3 artifacts.report_completedfired beforerun_postprocessindo_sweep.run()'sfinallyblock, so thecompletedPUT reached the collector before any rollup was generated.StatusReporter.report_artifacts(logs_url)for eager mid-run pushes — fires as soon as S3 sync completes, before AI analysis runs, so a hanging or crashing analyzer cannot strand the artifact pointer.Changes
src/srtctl/core/status.pyreport_completed(exit_code, logs_url=None)kwarg; newreport_artifacts(logs_url)that PUTs atstatus=benchmark(does not flip lifecycle)src/srtctl/cli/mixins/postprocess_stage.pyrun_postprocess(exit_code, reporter=None); eagerreporter.report_artifactsafter S3 sync; stashself._last_logs_url; removed the broken_report_metricssrc/srtctl/cli/do_sweep.pyfinallyreordered sorun_postprocessruns beforereport_completed; passesreporterin and pullslogs_urlback for the final PUTtests/test_status.pylogs_urlkwarg andreport_artifacts(disabled, empty URL, success without flipping tocompleted)tests/test_postprocess.py_report_metricsassertions with the eager-push + stash shape; added skip-paths for no-S3-URL and no-reporter casessrc/srtctl/core/config.py,src/srtctl/core/__init__.pyfind_cluster_config_path, addget_cluster_aliasesWhy two PUTs
Defense in depth: the eager
report_artifactsPUT fires the moment S3 sync returns, independent of whether the AI analyzer runs, times out, or crashes. The finalreport_completedindo_sweepthen reasserts the samelogs_urlidempotently atstatus=completed. If either PUT succeeds, the collector has the pointer.Compat
report_completed(exit_code)still works —logs_urlis optional. No call-site changes required for existing callers.logs_urlhas always been a first-class field inJobUpdatePayload.Test plan
uv run pytest tests/ -q— 589 passed, 2 skipped, 6 deselecteduv run ruff checkon touched files — all passuv run ruff format --checkon touched files — all passreporting.status.endpointlands on a cluster withreporting.s3configured — expected:GET /api/jobs/{id}returnslogs_urlpointing at the S3 prefix.🤖 Generated with Claude Code