Skip to content

Commit 5c23404

Browse files
authored
[CI] Add SHA256 checksum for binary uploads to S3 (#6172)
This ensures binary uploads to S3 attaches a SHA256 checksum. I am not familiar with this code base, but from what I can gather the S3 uploads seem to be missing SHA256 for various wheel file URLs. This seem to lead to missing wheel hashes at https://download.pytorch.org/. My guess is that the checksum is not available when it is retrieved [when the html is generated](https://github.com/pytorch/test-infra/blame/a797a36aab02198636cb2d032f24b894d668cefe/s3_management/manage.py#L535). With this change, it should ensure that the checksum is calculated before upload and attached to the object. Relates-to: pytorch/pytorch#76557
1 parent 7c97566 commit 5c23404

File tree

3 files changed

+5
-5
lines changed

3 files changed

+5
-5
lines changed

.github/actions/binary-upload/action.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ runs:
2929
# shellcheck disable=SC1090
3030
source "${BUILD_ENV_FILE}"
3131
32-
pip install awscli==1.32.18
32+
pip install awscli==1.37.0
3333
yum install -y jq
3434
3535
export AWS_ROLE_ARN="arn:aws:iam::749337293305:role/gha_workflow_nightly_build_wheels"
@@ -43,5 +43,5 @@ runs:
4343
| jq -r '.value' > "${AWS_WEB_IDENTITY_TOKEN_FILE}"
4444
4545
for pkg in dist/*; do
46-
aws s3 cp "$pkg" "${PYTORCH_S3_BUCKET_PATH}" --acl public-read
46+
aws s3 cp "$pkg" "${PYTORCH_S3_BUCKET_PATH}" --acl public-read --checksum-algorithm SHA256
4747
done

.github/workflows/_binary_upload.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -109,15 +109,15 @@ jobs:
109109
# shellcheck disable=SC1090
110110
source "${BUILD_ENV_FILE}"
111111
112-
pip install awscli==1.32.18
112+
pip install awscli==1.37.0
113113
114114
AWS_CMD="aws s3 cp --dryrun"
115115
if [[ "${NIGHTLY_OR_TEST:-0}" == "1" ]]; then
116116
AWS_CMD="aws s3 cp"
117117
fi
118118
119119
for pkg in dist/*; do
120-
${AWS_CMD} "$pkg" "${PYTORCH_S3_BUCKET_PATH}" --acl public-read
120+
${AWS_CMD} "$pkg" "${PYTORCH_S3_BUCKET_PATH}" --acl public-read --checksum-algorithm SHA256
121121
done
122122
123123
- name: Upload package to pypi

.github/workflows/update_disabled_tests.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ jobs:
112112
- name: Upload files to s3
113113
if: github.event_name != 'pull_request'
114114
run: |
115-
python3 -mpip install awscli==1.27.69
115+
python3 -mpip install awscli==1.37.0
116116
117117
aws s3 cp disabled-tests-condensed.json s3://ossci-metrics/disabled-tests-condensed.json
118118
aws s3 cp disabled-jobs.json s3://ossci-metrics/disabled-jobs.json

0 commit comments

Comments
 (0)