Fix the 127 test failures and 8 collection errors that CI never sees

[jeremymanning]

Part of #108 · Phase 1 · Depends on #110

Problem

When the suite is actually executed (in a purpose-built venv, since the documented install cannot start pytest — see #110), the verbatim result is:

127 failed, 1738 passed, 26 skipped, 229 warnings, 8 errors in 134.78s (0:02:14)

from 1898/2280 tests collected (382 deselected).

None of these are visible in CI, which runs 15 tests.

Failures grouped by root cause

N	Error	Interpretation
9	clustrix/executor_core.py:210: KeyError: 'manager'	executor refactor changed the job-dict shape; callers not updated
7	ValueError: Unsupported cluster type: local	config/executor drift — local is accepted by config but has no executor branch (executor_core.py:101)
6	TypeError: 'NoneType' object is not subscriptable
6	TypeError: __init__() missing 1 required keyword-only argument: 'response'	googleapiclient API changed under us — unpinned dep
5	ValueError: Unknown configuration parameter: partition / namespace / network_timeout / cleanup_on_failure	tests pass parameters ClusterConfig rejects
5	AttributeError: 'ClusterExecutor' object has no attribute '_get_k8s_result' / _get_k8s_error_log	tests reference deleted methods (test_kubernetes_integration.py:466,491,558)
4	AttributeError: 'ValidationCredentials' object has no attribute 'cred_manager'
3	AttributeError: 'EnhancedClusterConfigWidget' object has no attribute 'aws_region'
2	ModuleNotFoundError: No module named 'sklearn'	undeclared dep (see #110)
2	Failed: Timeout (>60.0s) at clustrix/kubernetes/aws_provisioner.py:763,771	real AWS retry loop (see #109)

Plus stale mock patch targets — the mocks no longer match the code:

unittest/mock.py:1378: AttributeError: <module 'clustrix.executor'> does not have the attribute 'setup_remote_environment'
unittest/mock.py:1378: AttributeError: <module 'clustrix.executor'> does not have the attribute 'cloudpickle'

Top failing files: test_notebook_magic.py (9), test_loop_analysis_advanced.py (6), test_kubernetes_integration.py (6), test_cloud_providers_huggingface_spaces.py (6), comprehensive/test_edge_cases_real.py (6), test_auth_fallbacks_real.py (5).

Important: triage before fixing

A large share of these are tests asserting against a code shape that no longer exists (deleted methods, renamed config params, changed dict keys). For each failure decide:

• Code is wrong -> fix the code.
• Test is wrong -> fix the test.
• Test is worthless (asserts a mock returns what the test told it to return) -> delete it; it is covered by the de-mock issue.

Per project policy, do not "simplify" a test to make it pass. Either the code is fixed so the existing test passes, or the test is deleted outright with a stated reason.

Acceptance criteria

• pytest tests/ -m "not real_world" -> 0 failed, 0 errors
• Every deletion is justified in the PR description
• googleapiclient and other volatile deps are version-pinned or adapted
• cluster_type: "local" either works end to end or is rejected at config time with a clear message — not accepted-then-crashed

Verification

pytest tests/ -m "not real_world" -o addopts="" -q --tb=no | tail -3

[jeremymanning]

Partial update from #130 (merged as #134): the collection errors are gone; the test failures are not.

Measured on master at 760432c in a clean [dev]-only venv:

$ pytest tests/ --collect-only -q | tail -1
1674 tests collected            # 0 errors

The 6 errors that were reachable were:

• tests/real_world/test_container_registry_comprehensive.py:571 — a backslash inside an f-string expression, a SyntaxError on every Python before 3.12 while requires-python is >=3.8. An AST parse across all 295 files under tests/ and clustrix/ confirms it was the only one.
• 5 modules importing numpy/pandas at module scope with neither declared in any extra. Both added to [dev].

Worth noting for this issue's framing: that syntax error is why CI "never saw" much of anything. Collection aborted with Interrupted: 1 error during collection before the run began, so a bare pytest executed zero tests rather than the suite anyone assumed.

Still open here. With collection now clean, the failures are visible and countable for the first time. A partial run of the newly-collectible tree showed 13 failures in the first 59 tests, so the "127 failures" figure needs re-deriving against master rather than carried forward — it was measured when 6 modules could not even be imported.

Suggested next step for this issue: re-run and re-baseline the failure count now that collection succeeds, then triage. Related: #133 (flake8 gate that can never pass, plus a real f-string bug in test_direct_gpu_detection.py) and #135 (fast_ci.yml is invalid YAML and has never run a job).