Fix flaky backend tests that build a bare TestClient - #13824
Open
hysts wants to merge 1 commit into
Open
Conversation
Three tests build a bare TestClient(app). Starlette only keeps one event loop across requests when the client is entered as a context manager; without the with block each request gets its own blocking portal, and that loop closes as soon as the response is complete. schedule_record_run files the record with create_task and record_run then awaits anyio.to_thread.run_sync, so a write that has not come back by the time the response is done is left pending on a loop that will never run again. The tests then sit out the full 30 second _wait_for and report "no record was written". The heartbeat task in test_queueing is stranded the same way, ending up neither cancelled nor done. Entering the clients as context managers keeps one portal alive for the whole test. Measured with CI's command, pytest -n auto -m "not flaky and not serial": test_history.py alone went from 4 of 5 runs failing to 5 of 5 clean, and the whole backend suite from 4 of 5 failing to 3 of 3 clean. Every failing run took 37 to 43 seconds because it waited out the poll; every run after the change took 7 to 17 seconds without entering the wait at all.
Collaborator
🪼 branch checks and previews
|
Collaborator
🦄 no changes detectedThis Pull Request does not include changes to any packages.__No changes detected. __
|
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The change is a targeted, low-risk test-only fix that correctly aligns TestClient usage with Starlette’s expected lifecycle to eliminate the described race.
Pull request overview
This PR reduces backend test flakiness by ensuring Starlette’s TestClient is used as a context manager so a single event loop/portal is preserved across multiple requests within each test (preventing background tasks from being stranded on a closed loop).
Changes:
- Update
test/test_history.pyfixtures/tests to createTestClient(app)viawith TestClient(app) as clientand yield the entered client. - Update
test/test_queueing.pyto enterTestClient(app)in the samewithstatement as theasyncio.create_taskpatch.
File summaries
| File | Description |
|---|---|
| test/test_queueing.py | Enters TestClient as a context manager to keep the portal/event loop alive across requests in the heartbeat test. |
| test/test_history.py | Enters TestClient as a context manager in recording-related fixtures/tests to prevent background recording tasks from being orphaned. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This was referenced Sep 5, 2026
hysts
marked this pull request as ready for review
September 5, 2026 02:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
test/test_history.py's two recording tests andtest/test_queueing.py::test_heartbeat_task_cancelled_after_stream_completesfail intermittently. All three drive the app through a bareTestClient(app), and starlette only keeps a single event loop across requests when the client is entered as a context manager. Without thewithblock each request gets its own blocking portal, and that loop closes as soon as the response is complete.schedule_record_runfiles the record withasyncio.get_running_loop().create_task(...)andrecord_runthen awaitsanyio.to_thread.run_sync(...), so when the offload has not come back in time the task is left pending on a loop that will never run again and the record is never written. The tests then sit out the full 30 second_wait_forand report "no record was written". The heartbeat task in the queueing test is stranded the same way, which is why it ends up neithercancelled()nordone(). Under uvicorn the loop outlives the request, so this only ever affected the tests.The fix is to enter the clients as context managers so one portal spans the whole test. Four places, nine lines, no library change.
Measured locally with CI's own command,
pytest -n auto -m "not flaky and not serial":test/test_history.pyaloneThe timing is the part worth keeping. Every failing run took 37 to 43 seconds, because it waited out the 30 second poll, and every run after the change took 7 to 17 seconds without entering the wait at all. That is what says the race is gone rather than just less frequent.
One caveat on evidence, since the three tests are not equally covered by the above. The two
test_history.pytests reproduce locally and the numbers are theirs.test_heartbeat_task_cancelled_after_stream_completesdoes not reproduce locally, since it passes in isolation with or without the change; it is included because it was observed failing on CI and builds its client the same way. The change to it is safe either way, but it rests on that rather than on a measurement.Closes: #13823
AI Disclosure
We encourage the use of AI tooling in creating PRs, but the any non-trivial use of AI needs be disclosed. E.g. if you used Claude to write a first draft, you should mention that. Trivial tab-completion doesn't need to be disclosed. You should self-review all PRs, especially if they were generated with AI.
🎯 PRs Should Target Issues
Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.
Not adhering to this guideline will result in the PR being closed.
Testing and Formatting Your Code
PRs will only be merged if tests pass on CI. We recommend at least running the backend tests locally, please set up your Gradio environment locally and run the backed tests:
bash scripts/run_backend_tests.shPlease run these bash scripts to automatically format your code:
bash scripts/format_backend.sh, and (if you made any changes to non-Python files)bash scripts/format_frontend.sh