Skip to content

Commit 49840da

Browse files
Upload gyp packages to S3 after building
After building native module packages (bcrypt, cld, unix-dgram, @datadog/pprof), upload them to s3://asana-oss-cache/node-gyp/v1/ in addition to the GitHub Release. This enables codez to fetch these packages via Bazel http_file instead of committing ~112 MB of tarballs to git, saving ~305 MB total per checkout (node18/node20 tarballs are dead code and will be deleted). Changes: - build-node-packages.yml: Add AWS OIDC auth + S3 upload step after release upload - stage_for_s3.bash: Separate packages_*.tar.gz before fibers loop to prevent them from being incorrectly mixed into the fibers archive Requires IAM role `push_node_gyp_packages` to be provisioned first (Asana/codez PR #388637). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 1ed9d2d commit 49840da

2 files changed

Lines changed: 39 additions & 0 deletions

File tree

.github/workflows/build-node-packages.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,14 +12,19 @@ on:
1212
jobs:
1313
build-packages:
1414
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
15+
permissions:
16+
id-token: write
17+
contents: write
1518
strategy:
1619
matrix:
1720
include:
1821
- platform: linux
1922
arch: x64
23+
bazel_arch: amd64
2024
runs_on: ubuntu-22.04
2125
- platform: linux
2226
arch: arm64
27+
bazel_arch: arm64
2328
runs_on: ubuntu-22.04-arm
2429
runs-on: ${{ matrix.runs_on }}
2530

@@ -75,3 +80,22 @@ jobs:
7580
files: packages_${{matrix.arch}}.tar.gz
7681
env:
7782
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
83+
84+
- name: Configure AWS credentials
85+
uses: aws-actions/configure-aws-credentials@v4
86+
with:
87+
aws-region: us-east-1
88+
role-to-assume: arn:aws:iam::403483446840:role/autogen_github_actions_beta_push_node_gyp_packages
89+
90+
- name: Upload packages to S3
91+
run: |
92+
NODE_MAJOR=$(echo "${{ env.NODE_VERSION }}" | sed 's/^v//' | cut -d. -f1)
93+
S3_KEY="node-gyp/v1/packages_${{ matrix.bazel_arch }}_node${NODE_MAJOR}.tar.gz"
94+
echo "Uploading packages_${{ matrix.arch }}.tar.gz to s3://asana-oss-cache/${S3_KEY}"
95+
aws s3 cp "packages_${{ matrix.arch }}.tar.gz" "s3://asana-oss-cache/${S3_KEY}" --acl public-read
96+
echo "SHA256: $(sha256sum packages_${{ matrix.arch }}.tar.gz | awk '{print $1}')"
97+
echo ""
98+
echo "Update tools_repositories.bzl with:"
99+
echo " name = \"node_gyp_packages_${{ matrix.bazel_arch }}_node${NODE_MAJOR}\","
100+
echo " urls = [\"https://asana-oss-cache.s3.us-east-1.amazonaws.com/${S3_KEY}\"],"
101+
echo " sha256 = \"$(sha256sum packages_${{ matrix.arch }}.tar.gz | awk '{print $1}')\","

stage_for_s3.bash

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,21 @@ echo "Current timestamp is $TIMESTAMP"
1010
gh release download -p "*.gz"
1111
gh release download -p "*.xz"
1212

13+
# Separate packages tarballs — these are uploaded to S3 by the build-node-packages.yml
14+
# workflow and consumed by Bazel via http_file in codez. They should NOT be mixed into
15+
# the fibers archive.
16+
echo ""
17+
echo "=== Native packages (node-gyp) ==="
18+
echo "These are uploaded to s3://asana-oss-cache/node-gyp/ by the build-node-packages.yml workflow."
19+
for pkg in packages_*.tar.gz; do
20+
if [ -f "$pkg" ]; then
21+
echo " $pkg: sha256=$(sha256sum "$pkg" | awk '{print $1}')"
22+
rm "$pkg"
23+
fi
24+
done
25+
echo "No manual action needed for packages — they are already in S3."
26+
echo ""
27+
1328
curl "https://asana-oss-cache.s3.us-east-1.amazonaws.com/node-fibers/fibers-5.0.4.pc.tgz" --output fibers-5.0.4.tar.gz
1429
tar -xzf fibers-5.0.4.tar.gz
1530

0 commit comments

Comments
 (0)