test(dashboard-api): cover unhealthy/warnings/retry/catalog regression gaps#1101
Draft
yasinBursali wants to merge 1 commit intoLight-Heart-Labs:mainfrom
Draft
test(dashboard-api): cover unhealthy/warnings/retry/catalog regression gaps#1101yasinBursali wants to merge 1 commit intoLight-Heart-Labs:mainfrom
yasinBursali wants to merge 1 commit intoLight-Heart-Labs:mainfrom
Conversation
…n gaps Pytest additions across test_extensions.py and test_host_agent.py: - TestExtensionLifecycleStatus::test_stale_started_unhealthy — locks in the stale-window -> "unhealthy" transition for user extensions in _compute_extension_status. Uses now-305s for the progress fixture so the file falls through the started-recency window without tripping _read_progress's >3600s suppression. - TestEnableExtensionHookReturnHandling: two new cases for multi-svc mixed-outcome warnings — dep pre_start failure does not accumulate; main post_start failure is attributed to main only. - TestEnableRetryEdgeCases: three cases covering no-hook retry, corrupt progress JSON falling back to sync path, and mid-install setup_hook status routing through sync. Class name avoids collision with the TestEnableRetry class shipping in another in-flight branch. - TestModelDownloadCatalogUnavailable: three cases asserting 500 for missing catalog, 500 for corrupt catalog, and 403 preserved for clean catalog with model not listed. 5 of 9 new tests fail-as-expected on current upstream/main; they pass once the prerequisite branches (host-agent enable-retry; host-agent runtime-hygiene) land. PR is filed DRAFT for that reason. No production code touched. 12 pre-existing baseline failures in the full suite (Python 3.14 asyncio compat) are unchanged.
Collaborator
|
Merge-pass recheck: holding this as draft/red. The regression coverage is valuable, but the PR body says 5 tests are fail-as-expected until prerequisite branches land, and the live |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds pytest coverage for four dashboard-api / host-agent behaviors that were previously untested:
_compute_extension_status's stale-window →"unhealthy"transition for user extensions.enable_extension's multi-service mixed-outcome warnings list (cross-svc accumulator)._handle_model_downloadregression shield for the missing/corrupt catalog → 500 contract.Why
enable_extensionaccumulates per-svc warnings across multi-service enables, but only single-svc cases were tested. A regression that broke cross-svc warning attribution (warning gets attributed to the wrong svc, or main's failure spuriously suppresses dep warnings) would have shipped silently.TestEnableRetryclass shipping in another in-flight PR: extensions without apost_installhook, progress files that became corrupt mid-install, and extensions retrying while still in asetup_hookmid-install state.How
tests/test_extensions.py(+185 lines, 3 new test methods):TestExtensionLifecycleStatus::test_stale_started_unhealthy— fixture usesnow-305sforupdated_at(must be >300s to fall through thestarted-recency window AND <3600s to avoid_read_progresssuppressing the file as stale-error-only). Asserts the unhealthy path returns"unhealthy"for user extensions whose container is stale-started + reporting unhealthy.TestEnableExtensionHookReturnHandling::test_multi_svc_pre_start_failure_on_dep_does_not_warn— dep's pre_start fails, main's all pass; assertswarnings == [](terminal pre_start failure isn't accumulated).TestEnableExtensionHookReturnHandling::test_multi_svc_post_start_failure_on_main_only_warns_main— main's post_start fails, dep all pass; asserts exactly one warning naming main, none for dep.tests/test_host_agent.py(+285 lines, 2 new test classes):TestEnableRetryEdgeCases(named to avoid collision with the in-flightTestEnableRetryclass) — three cases for the retry path that exercise no-hook, corrupted-JSON-progress, and mid-installsetup_hookstates.TestModelDownloadCatalogUnavailable— three cases asserting 500 for missing catalog, 500 for corrupt catalog, and 403 preserved for clean catalog with model not listed.All tests use existing helper patterns (
_setup_user_ext,_patch_mutation_config, in-processHTTPServer(("127.0.0.1", 0), _mod.AgentHandler)wire pattern,monkeypatchfor module attribute changes) and run intmp_pathto avoid global state pollution.Testing
On
upstream/main(current state):AttributeErroron_read_progress_status(TestEnableRetryEdgeCases — the helper is added by another in-flight PR).403 != 500(TestModelDownloadCatalogUnavailable cases a and b — the 500 split lands in another in-flight PR).After the prerequisite branches merge, the rebase will turn all 9 tests green.
Pre-existing 12 baseline failures in the full pytest suite (Python 3.14 / asyncio compat) verified unchanged via stash-pop on the PR-Y worktree — none introduced by this PR.
Review
Critique Guardian: APPROVED. Falsifiability verified across all 9 cases (assertions on response shape, content, and warnings-list contents — not just status codes).
Known Considerations
TestEnableRetryEdgeCases(this PR) vsTestEnableRetry(other branch).pre_start+ mainpost_startfailures combined into one call. Edge-of-edge case; could be a future addition.Platform Impact
GPU_BACKEND-driven branching.