Report torch/torchvision versions and binary sizes for every validated build - #8613
Report torch/torchvision versions and binary sizes for every validated build#8613atalman wants to merge 3 commits into
Conversation
…d build check_wheel_size only measures linux and linux-aarch64 pip wheels, skips ROCm and libtorch, and exists to enforce a ceiling -- so windows, macos and ROCm builds report no size at all today. Add a per-build report that runs on every OS and never fails the job. It prints two sizes, because they answer different questions: the compressed wheel download size (what users pull from the index, read off pip's output) and the unpacked installed size (measured from the installed package, so it is available on every install path including uv/wheel-variants). Versions come from the installed packages rather than the matrix, so a mismatch between what was requested and what pip resolved is visible. The wheel-size parsing is factored out of check_wheel_size into parse_wheel_size_mb and reused; the ceiling check keeps its current scope and behaviour.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
meta-cla
Bot
added
the
CLA Signed
| report=$(cd "${TMPDIR:-/tmp}" 2>/dev/null || cd "${HOME}"; "${PYTHON_RUN}" - \ | ||
| "${TARGET_OS}" "${MATRIX_PYTHON_VERSION:-?}" "${MATRIX_GPU_ARCH_TYPE:-cpu}" \ | ||
| "${MATRIX_GPU_ARCH_VERSION:-}" "${torch_wheel_mb}" "${vision_wheel_mb}" <<'PY' | ||
| import os |
There was a problem hiding this comment.
would be better to have this as a standalone script, so lint and other rules would apply and it would be easier to maintain the quality.
There was a problem hiding this comment.
Agreed, done in 3edf8f3 — it is now .github/scripts/build_report.py.
That puts it under FLAKE8 and PYFMT (**/*.py) and MYPY (.github/scripts/**/*.py). mypy.ini gets one exemption, following the existing per-module sections in that file:
[mypy-.github.scripts.build_report]
# torch/torchvision are not installed in the lint environment; the script
# imports them defensively at runtime and degrades when they are absent.
disable_error_code = import-not-found, import-untypedOne bonus that only became clear while moving it: the heredoc version had to cd out of the repo before running python, because python - puts the cwd on sys.path and the validation runs from the pytorch/pytorch checkout — so import torch could pick up the source tree instead of the installed package. A script file makes sys.path[0] the script's own directory, so that hazard is gone and the cd with it.
validate_binaries.sh is 73 lines lighter; the shell function now just invokes the script and handles the job-summary append.
Per review: embedded in a bash heredoc the python was invisible to flake8, mypy and pyfmt. As .github/scripts/build_report.py it is covered by all three (mypy.ini gets an import-not-found exemption, since torch is not installed in the lint environment). Being a real script file also removes the reason the caller had to cd out of the repo first: sys.path[0] is now the script's own directory rather than the cwd, so `import torch` can no longer pick up the pytorch source checkout the validation runs from.
The [mypy-.github.scripts.build_report] section never matched: with no __init__.py in that directory mypy names the module build_report, not the dotted path, so import-not-found was still reported for torch and torchvision. Use ignore_missing_imports on the imported modules instead, which is how mypy resolves third-party modules that are absent from the lint environment. Verified with mypy 1.13.0 (the pinned version) against mypy.ini: clean.
2 participants
Every validated build now reports what it installed and how big it is.
Why
check_wheel_sizeexists to enforce an 850 MB ceiling, and its scope is deliberately narrow: linux and linux-aarch64 pip wheels only, skipping ROCm and libtorch. That means windows, macos and ROCm builds report no size at all, and even where the check does run the number only appears as an annotation.What this adds
A
write_build_reportstep after the smoke tests, on every OS, that never fails the job:Two sizes, because they answer different questions:
Downloading/Using cachedline, so it is-on the uv/wheel-variants path and when the wheel was already satisfied.Versions are read from the installed packages rather than from the matrix, so a mismatch between what was requested and what pip resolved shows up in the report.
The report goes to stdout always, and is appended to the job summary when the runner exposes a writable one -- it is not reachable from inside the validation container, so that append is guarded rather than assumed.
Refactor
The size parsing is factored out of
check_wheel_sizeintoparse_wheel_size_mb <log> <dist>and reused for both torch and torchvision.check_wheel_sizekeeps its current scope, threshold and behaviour; only the parsing moved.Not covered
validate_libtorch.py), so they still report nothing. Worth a follow-up if libtorch size matters.Test plan
bash -nclean.parse_wheel_size_mbunit-checked against a captured pip log:torch->812.4,torchvision->8.1, absent distribution -> empty (torch-[0-9] does not match torchvision-/torchaudio-).91002 -> 9.10.2,92400 -> 9.24.0,91701 -> 9.17.1, matching the values published in pytorch RELEASE.md.write_build_reportwith no torch installed rendersimport failed: ...and exits 0 rather than aborting the job.GITHUB_STEP_SUMMARYpointing at an unwritable path, the function emits zero bytes on stderr and still exits 0.