Fix: pytest never exits after the summary, wedged by abandoned non-daemon threads

Every CI job printed its pytest summary and then sat there until the
fifteen-minute cap killed it -- 5m41s of dead time on ubuntu/3.11, and the
same on all eight matrix jobs.

Root cause: tests/comprehensive/test_edge_cases_real.py::test_deadlock_prevention
builds a deliberate AB-BA lock inversion in two *non-daemon* threads, joins
them with timeout=5, and abandons them when (as designed) they deadlock.
Py_FinalizeEx calls threading._shutdown(), which joins every surviving
non-daemon thread with no timeout, so the interpreter could never finalize.
The test is also re-run by reflection from test_comprehensive_edge_case_suite,
so four threads were wedged, not two.

Evidence, from a faulthandler dump armed at pytest_unconfigure:

  AT-UNCONFIGURE non-daemon alive: 4
    ALIVE Thread-6 target=...potential_deadlock.<locals>.worker1
    ALIVE Thread-7 target=...potential_deadlock.<locals>.worker2
    ALIVE Thread-8 target=...potential_deadlock.<locals>.worker1
    ALIVE Thread-9 target=...potential_deadlock.<locals>.worker2
  Thread 0x33da2b000: test_edge_cases_real.py line 635 in worker2
  Thread 0x33ca1f000: test_edge_cases_real.py line 629 in worker1
  Thread 0x33ba13000: test_edge_cases_real.py line 635 in worker2
  Thread 0x33aa07000: test_edge_cases_real.py line 629 in worker1
  Thread 0x1fbd91d80: threading.py line 1477 in _shutdown

Marking those two threads daemon changes nothing the test observes -- the
deadlock still happens, the joins still time out, is_alive() is still True and
the assertions are untouched -- but abandoning them no longer wedges the
process.

Same defect, second site: tests/real_world/conftest.py::_within claimed to
abandon its worker, but a ThreadPoolExecutor's workers are non-daemon and
concurrent.futures joins all of them, untimed, at interpreter exit even after
shutdown(wait=False). Abandoning a seventy-second DNS lookup only moved the
wait to process exit. Replaced with an actual daemon thread, preserving the
existing semantics (OSError -> None, timeout -> None, anything else re-raised).

Isolating measurement, tests/comprehensive/test_edge_cases_real.py alone:
  before: summary at 59.98s, still not exited 90s later
  after:  summary at 62.60s, exited 0.66s later

Full CI selection:
  before: [345.05] 1764 passed ... -- never exited, killed after 6 minutes
  after:  [343.58] 1764 passed ... / [345.64] EXITED rc=0

Author: jeremymanning

tests/comprehensive/test_edge_cases_real.py

@@ -635,9 +635,19 @@ def worker2():
                     with lock1:
                         results.append("worker2")
 
-            # Use timeout to prevent actual deadlock
-            t1 = threading.Thread(target=worker1)
-            t2 = threading.Thread(target=worker2)
+            # These two workers take their locks in opposite orders, so the
+            # deadlock below is the point of the test, not an accident: the
+            # join timeout is what keeps *this* function from hanging.
+            #
+            # They must be daemons. The joins below give up after five
+            # seconds and abandon threads that are wedged forever, and
+            # `threading._shutdown()` joins every surviving non-daemon thread
+            # with no timeout before the interpreter can finalize. Leaving
+            # these non-daemon wedged the whole pytest process after the
+            # summary line was printed -- every CI job burned its remaining
+            # budget there and was killed at the fifteen-minute cap.
+            t1 = threading.Thread(target=worker1, daemon=True)
+            t2 = threading.Thread(target=worker2, daemon=True)
 
             t1.start()
             t2.start()

tests/real_world/conftest.py

@@ -6,9 +6,9 @@
 import pytest
 from pathlib import Path
 import tempfile
-import concurrent.futures
 import functools
 import socket
+import threading
 
 from tests.real_world import RealWorldTestManager, TestCredentials, TempResourceManager
 
@@ -32,17 +32,32 @@ def _within(seconds, func, *args):
     The resolver calls here are not interruptible, so the worker thread is left
     to finish on its own; it is a daemon and holds nothing the caller needs.
     """
-    # Deliberately not a `with` block: its __exit__ calls shutdown(wait=True)
-    # and blocks until the worker finishes, which defeats the timeout entirely
-    # -- a call that should have been abandoned after three seconds still took
-    # thirty.
-    pool = concurrent.futures.ThreadPoolExecutor(max_workers=1)
-    try:
-        return pool.submit(func, *args).result(timeout=seconds)
-    except (concurrent.futures.TimeoutError, OSError):
-        return None
-    finally:
-        pool.shutdown(wait=False)
+    # A plain daemon thread, not a ThreadPoolExecutor. The executor's workers
+    # are non-daemon, and `concurrent.futures` joins every one of them --
+    # untimed -- on the way out of the interpreter, even after
+    # `shutdown(wait=False)`. Abandoning a seventy-second lookup that way only
+    # moved the wait from here to process exit. A daemon thread is genuinely
+    # abandonable: nothing joins it and the interpreter does not wait for it.
+    outcome = {}
+
+    def call():
+        try:
+            outcome["value"] = func(*args)
+        except BaseException as exc:  # re-raised below, in the caller's thread
+            outcome["error"] = exc
+
+    worker = threading.Thread(target=call, daemon=True)
+    worker.start()
+    worker.join(seconds)
+
+    error = outcome.get("error")
+    if error is not None:
+        # Unresolvable names are the expected off-network answer, not a fault.
+        if isinstance(error, OSError):
+            return None
+        raise error
+    # Absent on timeout, because the worker never got as far as storing one.
+    return outcome.get("value")
 
 
 @functools.lru_cache(maxsize=1)