Skip to content

Commit 2821290

Browse files
committed
fix(test): stop telemetry bootstrap tests blocking on real OTel exporter under py3.14
Tests in TestConfigureTelemetryIdempotency and related classes were hanging ~277 s each on Python 3.14 (just under the 300 s pytest --timeout), causing the CI unit leg to take ~40 min instead of 7-19 min. Root cause: the _reset_otel_globals autouse fixture captured the current global MeterProvider (which is _PROXY_METER_PROVIDER, the OTel process-wide proxy singleton) and attempted to "restore" it in teardown by calling set_meter_provider(_PROXY_METER_PROVIDER). OTel's set_meter_provider is intentionally a one-shot operation guarded by a Once object. When no real provider had yet been installed (because the test mocked set_meter_provider at the opentelemetry.metrics namespace), the Once gate had not yet fired, so the teardown call fired it for the first time with the proxy as its own argument. This triggered: _PROXY_METER_PROVIDER.on_set_meter_provider(_PROXY_METER_PROVIDER) → acquires _PROXY_METER_PROVIDER._lock → iterates proxy meters, calls meter.on_set_meter_provider(proxy) → _ProxyMeter.on_set_meter_provider calls proxy.get_meter() → _PROXY_METER_PROVIDER.get_meter() tries to acquire _PROXY_METER_PROVIDER._lock → deadlock (non-reentrant Lock, blocks forever) On Python 3.12 this was order-dependent: if TestConfigureTelemetryEnabled ran first, its teardown fired the Once gate harmlessly, making all later teardowns no-ops. On Python 3.14, CI runs these test classes in isolation (per-class or per-file grouping changed by bumping python.default_version 3.12→3.14), so the gate fires in teardown of the very first test that mocked set_meter_provider, hitting the deadlock deterministically every time. Fix: remove the restore logic from _reset_otel_globals entirely. OTel's global meter provider is a process-wide Once-guarded singleton; it is not designed to be reset between tests. The module-level _state object (_reset_telemetry_state) is the correct isolation boundary for unit tests of idempotency/flag logic. Tests that need to intercept set_meter_provider already mock it at the call site (which they do).
1 parent 0c66fb7 commit 2821290

1 file changed

Lines changed: 17 additions & 16 deletions

File tree

tests/unit/infrastructure/di/test_telemetry_bootstrap.py

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -59,29 +59,30 @@ def _make_container_with_otel(otel_config: OtelConfig) -> DIContainer:
5959
def _reset_otel_globals():
6060
"""Reset telemetry state before and after every test for isolation.
6161
62-
Also restores the global OTel MeterProvider so that enabled=True tests
63-
installing a real provider do not leak into subsequent tests.
62+
OTel's set_meter_provider() is intentionally a one-shot operation
63+
(guarded by a Once gate): calling it a second time is a no-op. Trying
64+
to "restore" the original proxy provider after each test by calling
65+
set_meter_provider(_PROXY_METER_PROVIDER) is therefore wrong — if the
66+
Once gate has not yet fired it fires with the proxy as its own argument,
67+
causing _ProxyMeterProvider.on_set_meter_provider(_PROXY_METER_PROVIDER)
68+
to hold _ProxyMeterProvider._lock and then call back into
69+
_ProxyMeterProvider.get_meter(), which tries to acquire the same lock a
70+
second time and deadlocks indefinitely (seen as ~277 s hangs under
71+
Python 3.14, where the deadlock is reliably deterministic because
72+
threading.Lock acquire now always blocks without a timeout by default).
73+
74+
The correct approach for unit tests that only test idempotency flags and
75+
module-level _state is to reset _state via _reset_telemetry_state() and
76+
to mock opentelemetry.metrics.set_meter_provider at the call sites that
77+
need it (which the enabled-path tests already do). We must not attempt
78+
to mutate OTel's process-wide Once-guarded singleton in teardown.
6479
"""
6580
telemetry_module._reset_telemetry_state()
66-
# Capture the current global meter provider so we can restore it.
67-
try:
68-
from opentelemetry import metrics as _otel_metrics
69-
70-
_original_meter_provider = _otel_metrics.get_meter_provider()
71-
except (ImportError, Exception):
72-
_original_meter_provider = None
7381

7482
yield
7583

7684
telemetry_module._reset_telemetry_state()
7785

78-
# Restore the global meter provider to what it was before the test.
79-
if _original_meter_provider is not None:
80-
with suppress(Exception):
81-
from opentelemetry import metrics as _otel_metrics
82-
83-
_otel_metrics.set_meter_provider(_original_meter_provider)
84-
8586

8687
# ---------------------------------------------------------------------------
8788
# Test: enabled=False is a no-op

0 commit comments

Comments
 (0)