Skip to content

[Deprecation] Make memmap pickle loading explicit - #1763

Merged
vmoens merged 2 commits into
pytorch:mainfrom
peterdsharpe:agent/memmap-pickle-loading
Aug 14, 2026
Merged

[Deprecation] Make memmap pickle loading explicit#1763
vmoens merged 2 commits into
pytorch:mainfrom
peterdsharpe:agent/memmap-pickle-loading

Conversation

@peterdsharpe

Copy link
Copy Markdown
Contributor

Summary

  • expose an explicit pickle policy across the memmap load and refresh APIs
  • preserve compatibility in 0.13 with a warning ahead of the 0.14 default change
  • preserve trusted save round-trips and remove obsolete pickle sidecars when metadata no longer needs them

User impact

Callers can reject pickle while loading less-trusted artifacts, while workflows that deliberately store arbitrary Python metadata can opt in explicitly. Existing callers receive a transition warning only when a pickle sidecar is encountered.

Root cause

Non-tensor values deliberately fall back to pickle when JSON cannot represent them, but the corresponding load path did not expose a caller-selectable policy. Reusing a save directory could also retain an obsolete sidecar after the value became JSON-compatible.

Validation

  • 341 relevant tests passed, with 3 expected skips
  • 33 focused policy, execution-boundary, overwrite, and slow multiprocessing tests passed against the isolated commit
  • ufmt, Black, flake8, pydocstyle, RST title, and diff checks passed

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 10, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation Test tensorclass Deprecation Announces or enacts a deprecation labels Aug 10, 2026
@vmoens
vmoens marked this pull request as ready for review August 14, 2026 10:29

@vmoens vmoens left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

@vmoens
vmoens merged commit 2a7c3d4 into pytorch:main Aug 14, 2026
47 of 48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Deprecation Announces or enacts a deprecation documentation Improvements or additions to documentation tensorclass Test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants