Skip to content

test: rewrite stale-keepalive retry test without real socket dependence#611

Merged
dionhaefner merged 2 commits into
mainfrom
jpb/fix-keepalive-test
May 27, 2026
Merged

test: rewrite stale-keepalive retry test without real socket dependence#611
dionhaefner merged 2 commits into
mainfrom
jpb/fix-keepalive-test

Conversation

@jpbrodrick89

Copy link
Copy Markdown
Contributor

Summary

`test_stale_keepalive_connection_is_handled` was failing intermittently — specifically on `tests-base (ubuntu-latest, 3.14, latest)` — with `assert 1 == 2` (only one call to `session.request`, not the expected two-call retry sequence).

The original test was integration-style: it spun up a real uvicorn server with `timeout_keep_alive=0`, established a connection, slept 100 ms to give the server time to FIN, monkeypatched urllib3's `is_connected` property to lie once, then asserted that the resulting `ConnectionError` caused `HTTPClient` to retry.

On Linux + Python 3.14 + latest deps, the first request succeeded instead of raising — either uvicorn's FIN didn't arrive in the 100 ms window on that platform, or newer urllib3 transparently refreshed the stale connection before `requests.Session.request` returned. Either way, the retry path never fired and `mock.call_count` came in at 1 rather than 2. (macOS 3.14 latest and Linux 3.10 latest both still pass.)

That brittleness is a symptom of the test exercising real-socket timing and urllib3 internals, not `HTTPClient`'s actual contract. The contract is simply: "if `session.request` raises `ConnectionError`, retry once". Inject the `ConnectionError` at the session layer directly — same assertion (`call_count == 2`, correct result returned), no environmental dependence.

Test plan

  • `pytest tests/sdk_tests/test_tesseract.py::test_stale_keepalive_connection_is_handled` passes locally
  • Sibling timeout/retry tests in `test_tesseract.py` still pass
  • `tests-base (ubuntu-latest, 3.14, latest)` should now stop failing on this test

🤖 Generated with Claude Code

The original integration-style test spun up a real uvicorn server with
timeout_keep_alive=0, established a connection, slept 100 ms to give
the server time to close the socket, monkeypatched urllib3's
is_connected property to lie once, then asserted the resulting
ConnectionError caused HTTPClient to retry.

The assertion was flaky on Linux + Python 3.14 + latest deps, where
either the FIN didn't arrive in 100 ms reliably or newer urllib3
transparently refreshed the stale connection — in both cases the first
request succeeded without raising, so HTTPClient's retry never fired
and mock.call_count came in at 1 instead of 2.

That brittleness is testing real-socket timing and urllib3 internals,
not HTTPClient's contract. The contract is just: "if session.request
raises ConnectionError, retry once". Inject the ConnectionError at the
session layer directly — same assertion, no environment dependence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented May 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.11%. Comparing base (72523bd) to head (04ef4df).

❗ There is a different number of reports uploaded between BASE (72523bd) and HEAD (04ef4df). Click for more details.

HEAD has 28 uploads less than BASE
Flag BASE (72523bd) HEAD (04ef4df)
35 7
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #611      +/-   ##
==========================================
- Coverage   77.08%   67.11%   -9.97%     
==========================================
  Files          32       32              
  Lines        4495     4495              
  Branches      739      739              
==========================================
- Hits         3465     3017     -448     
- Misses        728     1237     +509     
+ Partials      302      241      -61     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jpbrodrick89

Copy link
Copy Markdown
Contributor Author

@dionhaefner I appreciate this changes the test's behaviour a lot, there's a chance it was just flaky and a rerun might have allowed it to pass, let me know your preference how best you'd like this handled.

@dionhaefner dionhaefner left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine with me, thanks!

@dionhaefner
dionhaefner enabled auto-merge (squash) May 27, 2026 12:59
@dionhaefner
dionhaefner disabled auto-merge May 27, 2026 13:00
@dionhaefner dionhaefner changed the title fix: rewrite stale-keepalive retry test without real socket dependence test: rewrite stale-keepalive retry test without real socket dependence May 27, 2026
@dionhaefner
dionhaefner enabled auto-merge (squash) May 27, 2026 13:00
@dionhaefner
dionhaefner merged commit 51e681b into main May 27, 2026
23 checks passed
@dionhaefner
dionhaefner deleted the jpb/fix-keepalive-test branch May 27, 2026 13:05
@pasteurlabs pasteurlabs locked and limited conversation to collaborators May 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants