Skip to content

Commit 786526a

Browse files
committed
test(k8s): register retry classifier suite-wide to kill real retry-sleeps
The k8s handlers wrap kubernetes-client calls in the resilience retry layer; the K8sRetryClassifier (registered during provider bootstrap in production) marks 404/4xx ApiExceptions as non-retryable. Unit/mocked tests skip bootstrap, so terminal errors were retried with real 1s time.sleep backoff — a handful of tests dominated the suite. Add a tree-wide autouse fixture (tests/providers/k8s/ conftest.py) that registers the classifier, mirroring production. Also fix test_job_deleted_not_found_no_warning_flood to raise a real ApiException(404) instead of a bare Exception('Not Found') so the classifier recognises it and the handler fails fast — no is_not_found override or retry-backoff hack needed. k8s leg ~40s -> ~18s, 1764 passing, no behaviour change.
1 parent aa99279 commit 786526a

2 files changed

Lines changed: 48 additions & 4 deletions

File tree

tests/providers/k8s/conftest.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
"""Shared fixtures for the k8s provider test tree.
2+
3+
The k8s handlers wrap kubernetes-client calls in the resilience retry layer.
4+
Whether an error is retried is decided by the *retry classifier registry*: the
5+
``K8sRetryClassifier`` marks 404/400/401/403 (and similar terminal API errors)
6+
as non-retryable so the handler fails fast instead of retrying.
7+
8+
In production that classifier is registered during provider bootstrap
9+
(``K8sProviderStrategy.initialize`` → ``registration.py``). Unit and mocked
10+
tests construct handlers directly and never run that bootstrap, so without this
11+
fixture the registry is empty: every exception — including the 404s that
12+
mocked tests deliberately provoke — is treated as retryable and retried with
13+
real ``time.sleep`` backoff (``base_delay`` defaults to 1s). A handful of
14+
tests then spend seconds sleeping through retries they should never attempt,
15+
dominating the k8s suite's wall-clock.
16+
17+
Registering the classifier for the whole k8s tree makes those terminal errors
18+
fail fast (matching production behaviour) and removes the spurious retry
19+
sleeps. It is autouse + function-scoped with a clean teardown so no test
20+
leaks classifier state into another.
21+
"""
22+
23+
from __future__ import annotations
24+
25+
import pytest
26+
27+
28+
@pytest.fixture(autouse=True)
29+
def _register_k8s_retry_classifier():
30+
"""Register K8sRetryClassifier for each k8s test (mirrors provider bootstrap)."""
31+
from orb.infrastructure.resilience.retry_classifier_registry import (
32+
clear_classifiers,
33+
register_retry_classifier,
34+
)
35+
from orb.providers.k8s.resilience.retry_classifier import K8sRetryClassifier
36+
37+
register_retry_classifier(K8sRetryClassifier())
38+
yield
39+
clear_classifiers()

tests/providers/k8s/unit/handlers/test_status_resolver_perf.py

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -716,7 +716,12 @@ def test_job_deleted_not_found_no_warning_flood(caplog: Any) -> None:
716716

717717
from orb.providers.k8s.infrastructure.handlers.job_handler import K8sJobHandler
718718

719-
not_found_exc = Exception("Not Found")
719+
# Raise a real ApiException(404): the K8sRetryClassifier recognises it as
720+
# non-retryable so the handler fails fast — no retry backoff, no sleep.
721+
from kubernetes.client.exceptions import ApiException
722+
723+
not_found_exc = ApiException(status=404)
724+
not_found_exc.status = 404
720725

721726
core_v1 = MagicMock()
722727
core_v1.list_namespaced_pod.return_value = SimpleNamespace(items=[])
@@ -731,9 +736,9 @@ def test_job_deleted_not_found_no_warning_flood(caplog: Any) -> None:
731736
logger_mock = MagicMock()
732737
handler = K8sJobHandler(kubernetes_client=client, config=config, logger=logger_mock)
733738

734-
# Make is_not_found return True for our exception.
735-
handler.is_not_found = lambda exc: True # type: ignore[method-assign]
736-
739+
# No is_not_found override or retry-backoff hack needed: the real
740+
# ApiException(404) is recognised as not-found by the handler and as
741+
# non-retryable by K8sRetryClassifier, so it fails fast on the first attempt.
737742
request = _req(provider_api="Job", deployment_name="job-1")
738743

739744
with caplog.at_level(logging.WARNING):

0 commit comments

Comments
 (0)