Skip to content

Shield file stream cleanup from cancellation - #1318

Open
Kuang-xianxin wants to merge 6 commits into
agronholm:masterfrom
Kuang-xianxin:fix/file-stream-cancellation-cleanup
Open

Shield file stream cleanup from cancellation#1318
Kuang-xianxin wants to merge 6 commits into
agronholm:masterfrom
Kuang-xianxin:fix/file-stream-cancellation-cleanup

Conversation

@Kuang-xianxin

@Kuang-xianxin Kuang-xianxin commented Sep 9, 2026

Copy link
Copy Markdown

NOTE Erasing or replacing the contents of this template will result in your pull

request being summarily closed without consideration!

Changes

Related to #1314.

When a cancelled task exits a FileReadStream or FileWriteStream context,
to_thread.run_sync() can raise before dispatching file.close(), leaving the
file open. Shield the shared close operation, then call
checkpoint_if_cancelled() after the file is closed so cancellation is delivered
by the close operation itself.

The regressions use public from_path() contexts and assert that closing both
releases the real file and propagates cancellation. They have no independent
checkpoint, scope-state or ordinary write/read-back assertions.

Validation on Python 3.13.14 / Windows:

  • Both regressions fail on the original implementation because files remain open,
    and on the shield-only version because close does not raise cancellation.
    Each comparison has 8 failures across asyncio, winloop, eager asyncio and Trio.
  • Final file-stream, AsyncFile and stapled-stream tests: 131 passed.
  • Non-mypy pre-commit checks and focused source/test mypy pass.
  • The full configured mypy hook has known Windows baseline diagnostics. The
    earlier wider file/path run had 372 passes, 96 skips and 76 Windows symlink
    failures/setup errors; all 76 also reproduced on the unchanged base. These
    platform-dependent checks are not claimed as passing.

The changelog records the fix; no user-guide addition is needed for expected
cleanup behavior.

Checklist

If this is a user-facing code change, like a bugfix or a new feature, please ensure that

you've fulfilled the following conditions (where applicable):

  • You've added tests (in tests/) which would fail without your patch

  • You've updated the documentation (in docs/), in case of behavior changes or new

features

  • You've added a new changelog entry (in docs/versionhistory.rst).

If this is a trivial change, like a typo fix or a code reformatting, then you can ignore

these instructions.

Updating the changelog

If there are no entries after the last release, use **UNRELEASED** as the version.

If, say, your patch fixes issue #123, the entry should look like this:


- Fix big bad boo-boo in task groups

  (`#123 <https://github.com/agronholm/anyio/issues/123>`_; PR by @yourgithubaccount)

If there's no issue linked, just link to your pull request instead by updating the

changelog after you've created the PR.

Comment thread docs/streams.rst Outdated

@agronholm agronholm left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added tests seem to also try to validate the cancellation mechanism. There are other tests to cover those, so these parts are just unnecessary noise here, agreed?

Comment thread tests/streams/test_file.py Outdated

assert scope.cancelled_caught
assert file.closed
assert file_path.read_bytes() == b"Hello"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a situation where the test would fail at this point?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed: once this real buffered file is closed, the read-back assertion only repeats ordinary write/close behavior covered elsewhere. In 261034c I removed that assertion and its payload write, along with the explicit checkpoint, pytest.fail() and cancelled_caught checks from both tests. Each regression now only triggers cancellation during context exit and asserts that the real file is closed.

The simplified tests still fail in all eight cases on the original implementation (asyncio, winloop, eager asyncio and Trio); the 131 related file-stream/AsyncFile/stapled-stream tests pass. The production fix is unchanged.

Comment thread src/anyio/streams/file.py
Comment on lines +41 to +42
with CancelScope(shield=True):
await to_thread.run_sync(self._file.close)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 4d8114e: the shielded file close completes first, then checkpoint_if_cancelled() delivers pending cancellation, matching the linked implementation.

The two existing regressions now expect cancellation from context exit itself and check that the real file was closed. They fail in all eight backend cases on the previous shield-only commit (no cancellation raised), and also on the original implementation (file left open). The 131 related file-stream/AsyncFile/stapled-stream tests and focused mypy pass; non-mypy pre-commit hooks pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants