Skip to content

Commit 09af7ad

Browse files
committed
[ci] Sign and notarize macOS releases with rcodesign directly
Sign and notarize by calling rcodesign directly instead of the indygreg/apple-code-sign-action: - The sign-publish job already installs rcodesign and calls it for encode-app-store-connect-api-key; the action only wrapped the sign and notarize steps, so this consolidates on one way of invoking the tool. - Drops a third-party action (personal namespace) that handled the PKCS#12 password and notarization API key. This repo SHA-pins third-party actions from non-vendor namespaces; indygreg/ was the only non-vendor action on a floating tag (GitHub actions/* and AWS aws-actions/* stay on version tags). Harden the release job: - Pin the rcodesign tarball to a repo-held SHA256 (APPLE_CODESIGN_SHA256 in the workflow, per-arch digests in the builder-image script) rather than trusting the release's own .sha256 sidecar, which a compromised release could replace alongside the tarball. - Restrict the push trigger to release tags (tags: ["v*"]) so a non-release tag push cannot sign and publish artifacts, matching build-linux-packages.yml. - Delete each signing secret immediately after its last use (P8 after encoding the API key, P12 after Apple-signing, GPG key after zip-signing, API JSON after notarization), so none remain on disk once the AWS credentials action and upload-artifact run. An ERR trap in the credential-materialization step removes the temp files if it fails before their paths are cleanup-tracked; the always() Cleanup remains the backstop for later steps. - Upload each zip's detached .sig to S3 before the zip itself, so a partial upload cannot leave a consumable zip without its signature. - Check out the requested tag on workflow_dispatch (ref:) and refuse to build/sign a non-tag ref, mirroring build-linux-packages.yml. Also: - Note in the workflow env block that the packaging Taskfile greps APPLE_CODESIGN_PKG_VERSION from this file. - Comment why executable bits are restored after artifact download, and that setup_gpg materializes the key at NFPM_SIGNING_KEY. - macos-zip.md: reword "load-bearing"/"Tier 1/2/3" jargon, fix a 404 notary link, note DEB now shares the GPG key, and rewrite the action-pinning section to reflect the direct-rcodesign approach.
1 parent b538ec4 commit 09af7ad

4 files changed

Lines changed: 243 additions & 142 deletions

File tree

.github/packaging/macos-zip.md

Lines changed: 95 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,8 @@ Per architecture, the macOS packaging pipeline produces:
3232
Each Mach-O binary inside a zip carries an Apple code signature with the
3333
`runtime` hardened-runtime flag. Each zip is signed with the project's GPG
3434
key and submitted to Apple's notary service before being published. The
35-
order is load-bearing; see *Apple-sign / GPG-sign / notarize order* in the
36-
maintenance notes.
35+
order of those three operations matters; see *Apple-sign / GPG-sign /
36+
notarize order* in the maintenance notes.
3737

3838
### Main entrypoints
3939

@@ -118,20 +118,31 @@ runner.
118118

119119
Apple signing, notarization, and publishing all run on Linux because:
120120

121-
- `indygreg/apple-code-sign-action@v1` is a Node JS action and runs on any
122-
runner that supports Node 20+
123-
- the underlying `rcodesign` tool is cross-platform and ships pre-built Linux
124-
binaries for both `x86_64` and `aarch64`
125-
- macOS GitHub-hosted minutes are roughly 10× the price of Linux minutes; the
121+
- `rcodesign` is cross-platform and ships pre-built Linux binaries for both
122+
`x86_64` and `aarch64`, so Apple-signing and notary submission never need a
123+
macOS host
124+
- macOS GitHub-hosted minutes are roughly 10x the price of Linux minutes; the
126125
parts of the pipeline that do not require a Mach-O build environment should
127126
not pay that premium
128127

129128
### Credential lifecycle
130129

131-
All Apple credentials and the GPG key are materialized into `mktemp` files
132-
and are explicitly removed by the workflow's `Cleanup` step (which runs
133-
`if: always()`). They are not echoed via `GITHUB_OUTPUT` and never reach
134-
either the GitHub Releases page or the `s3://${BUCKET}/macos/` prefix.
130+
All Apple credentials and the GPG key are materialized into `mktemp` files and
131+
each is deleted as soon as its last consumer finishes, so a given secret is on
132+
disk only across the steps that actually use it:
133+
134+
| Secret | Materialized in | Deleted after |
135+
| --- | --- | --- |
136+
| App Store Connect P8 | `Materialize Apple credentials` | encoding the API-key JSON (same step; its path is never exported) |
137+
| Signing PKCS#12 | `Materialize Apple credentials` | `Apple-sign binaries` |
138+
| GPG private key | `Import GPG key` | `Build zips and GPG-sign` |
139+
| Encoded API-key JSON | `Materialize Apple credentials` | `Notarize zips` |
140+
141+
By the time the AWS credentials action and `upload-artifact` run, no signing
142+
secret remains on disk. The `Cleanup` step (`if: always()`) is the backstop for
143+
failure paths that abort before a per-step delete. The credentials are not
144+
echoed via `GITHUB_OUTPUT` and never reach either the GitHub Releases page or
145+
the `s3://${BUCKET}/macos/` prefix.
135146

136147
The local validator mirrors this: ephemeral Apple PKCS#12, GPG private key,
137148
and ECDSA P8 are all written into a `mktemp -d` stage directory cleaned up by
@@ -165,8 +176,8 @@ produces an artifact whose internal entries are `avalanchego` and `subnet-evm`
165176
```
166177

167178
Downstream steps reference `build/avalanchego` and `build/subnet-evm`
168-
explicitly, so the `path: build` is load-bearing - using `path: .` would
169-
land the files at the workspace root and break the rest of the workflow.
179+
explicitly, so the `path: build` matters here - using `path: .` would land
180+
the files at the workspace root and break the rest of the workflow.
170181

171182
The production `build-zip-pkg.sh` then calls `7z a foo.zip build/avalanchego`,
172183
which preserves the `build/` prefix inside the zip. The validator therefore
@@ -179,10 +190,11 @@ The local validator (`build-macos-zip.sh`) invokes the production
179190
therefore exercises the same zip + GPG-sign code that runs in CI, not a copy.
180191
Behavioral divergence between local and CI is structurally avoided.
181192

182-
The Apple-sign and notarize steps are not yet wrapped this way because they
183-
live inside the `indygreg/apple-code-sign-action@v1` Node JS action rather
184-
than in a separate script; the local validator invokes `rcodesign` directly
185-
with the same flag shapes the action uses.
193+
The Apple-sign, notarize, and API-key-encode steps call `rcodesign` directly
194+
in both places (CI's `sign-publish` job and the local validator), with the
195+
same flags. CI installs `rcodesign` from the `indygreg/apple-platform-rs`
196+
release; the validator bakes it into its Docker image. Neither path depends
197+
on a third-party GitHub Action to reach the signing tool.
186198

187199
## Maintenance notes
188200

@@ -192,41 +204,50 @@ Apple's `codesign` only runs on macOS. The sign-publish job runs on Linux for
192204
cost reasons (see *Two-job split* above), so the signing tool must work on
193205
Linux. `rcodesign` (from `indygreg/apple-platform-rs`) is a pure-Rust
194206
re-implementation of Apple code-signing and notary submission that works
195-
without macOS. It is the same tool wrapped by
196-
`indygreg/apple-code-sign-action`, so using `rcodesign` directly in the local
197-
validator and via the action in CI keeps the underlying signing
198-
implementation consistent across both paths.
199-
200-
### Why `indygreg/apple-code-sign-action@v1` is currently pinned by major tag
201-
202-
The action is currently pinned to the floating `v1` tag rather than a full
203-
commit SHA. The pragmatic trade-off:
204-
205-
- `v1` follows the same pinning pattern as other GitHub Actions in this
206-
repository's release workflows (`actions/checkout@v5`,
207-
`aws-actions/configure-aws-credentials@v6`, etc.). Internal consistency.
208-
- A retag of `v1` to a malicious commit would let an attacker exfiltrate the
209-
PKCS#12 password and notarization API key. The action is on a personal
210-
GitHub namespace (`indygreg/`), which is meaningfully higher supply-chain
211-
risk than first-party `actions/*` and `aws-actions/*` pins.
212-
213-
The tarball that `indygreg/apple-platform-rs` ships is already SHA256-pinned
214-
in the local validator's Dockerfile (`Dockerfile.macos-zip`), via the
215-
`*.sha256` sidecar from the release. The CI workflow itself does not yet
216-
SHA-pin either the action or the rcodesign tarball it downloads inline.
217-
218-
When the upstream API stabilizes or supply-chain risk increases enough to
219-
justify the maintenance overhead, pin to a full commit SHA and enable
220-
Dependabot to update it.
207+
without macOS. Both the CI `sign-publish` job and the local validator invoke
208+
`rcodesign` directly, so the underlying signing implementation is identical
209+
across both paths.
210+
211+
### Why the signing tool is `rcodesign` invoked directly, not a GitHub Action
212+
213+
Earlier revisions of this workflow wrapped signing and notarization in the
214+
`indygreg/apple-code-sign-action` GitHub Action, a thin Node wrapper that
215+
downloads `rcodesign` and shells out to it. It was dropped in favor of
216+
calling `rcodesign` directly because:
217+
218+
- The `sign-publish` job already installs `rcodesign` (it needs it for
219+
`encode-app-store-connect-api-key`) and the local validator already calls
220+
it directly. Using the action for only the sign/notarize steps meant two
221+
different ways of invoking the same tool inside one job.
222+
- The action lives in a personal GitHub namespace (`indygreg/`) and handled
223+
the PKCS#12 password and notarization API key. A retag of its floating `v1`
224+
to a malicious commit would have let it exfiltrate those secrets. This
225+
repository SHA-pins third-party actions from non-vendor namespaces for
226+
exactly this reason (GitHub's own `actions/*` and AWS's `aws-actions/*` stay
227+
on version tags); `indygreg/` was the only non-vendor action on a floating
228+
tag.
229+
230+
The remaining actions in the sign-publish job are the vendor ones
231+
(`actions/checkout`, `actions/*-artifact`, `aws-actions/configure-aws-credentials`).
232+
The two that run after credentials are materialized (`configure-aws-credentials`
233+
and `upload-artifact`) run only after every signing secret has already been
234+
deleted at its last point of use (see *Credential lifecycle*).
235+
236+
Supply-chain integrity of the tool itself is handled by pinning the
237+
`rcodesign` release tarball's SHA256 in this repository, rather than trusting
238+
the release's own `*.sha256` sidecar (a compromised release could replace the
239+
tarball and the sidecar together). The CI workflow verifies against
240+
`APPLE_CODESIGN_SHA256`; the local builder image passes the per-arch digest as
241+
a Docker build arg (both live next to the pinned version and must be updated
242+
together on a version bump).
221243

222244
### Apple-sign / GPG-sign / notarize order
223245

224246
The CI workflow performs the signing operations in this order, and the
225-
order is load-bearing:
247+
order matters:
226248

227-
1. **Apple-sign each Mach-O binary** (`rcodesign sign` via the action). The
228-
signature is embedded inside the Mach-O via the `LC_CODE_SIGNATURE` load
229-
command.
249+
1. **Apple-sign each Mach-O binary** (`rcodesign sign`). The signature is
250+
embedded inside the Mach-O via the `LC_CODE_SIGNATURE` load command.
230251
2. **Zip each binary and detached-GPG-sign the resulting zip**
231252
(`build-zip-pkg.sh`). The GPG signature is over the zip bytes, which
232253
include the already-Apple-signed Mach-O.
@@ -261,20 +282,21 @@ This is a deliberate guard against a previously-latent failure mode:
261282
The precondition replaces "publish unsigned then error" with "fail at the
262283
import step with a clear message."
263284

264-
### Why `secrets.RPM_GPG_PRIVATE_KEY` is shared with RPM signing
285+
### Why `secrets.RPM_GPG_PRIVATE_KEY` is shared across package formats
265286

266-
The macOS zip workflow reuses the same GPG private key as the RPM packaging
267-
workflow rather than introducing a `MACOS_GPG_PRIVATE_KEY` secret.
287+
The macOS zip workflow reuses the same GPG private key as the RPM and DEB
288+
packaging workflows rather than introducing a `MACOS_GPG_PRIVATE_KEY` secret.
268289

269290
Rationale: the GPG key identifies *the project*, not the package format. End
270291
users who want to verify "is this artifact really from ava-labs?" use the
271-
same key whether they downloaded an RPM or a macOS zip (Linux tarballs are
272-
currently unsigned; if they ever gain GPG signing, the same key would
273-
naturally be reused). Maintaining two keys for the same provenance claim
274-
would create rotation and trust-chain complexity without a corresponding
275-
security benefit. The `RPM_GPG_PRIVATE_KEY` secret name predates macOS zip
276-
signing; renaming it would require coordinated changes across multiple
277-
workflows. The name is historical; the key is project-wide.
292+
same key whether they downloaded an RPM, a DEB, or a macOS zip (plain Linux
293+
tarballs are currently unsigned; if they ever gain GPG signing, the same key
294+
would naturally be reused). Maintaining separate keys for the same provenance
295+
claim would create rotation and trust-chain complexity without a
296+
corresponding security benefit. The `RPM_GPG_PRIVATE_KEY` secret name
297+
predates both DEB and macOS zip signing; renaming it would require
298+
coordinated changes across multiple workflows. The name is historical; the
299+
key is project-wide.
278300

279301
If the project ever needs per-format signing keys (for example, to support
280302
different rotation cadences), the migration path is: add new format-specific
@@ -306,8 +328,8 @@ boundary use the sec-team naming convention.
306328

307329
### Local validator scope
308330

309-
The local validator is intentionally "Tier 1": it exercises the parts of the
310-
pipeline that are reproducible offline. What it does and does not cover:
331+
The local validator intentionally covers only the parts of the pipeline that
332+
are reproducible offline. What it does and does not cover:
311333

312334
| Part | Local validator | Why |
313335
| --- | --- | --- |
@@ -316,12 +338,12 @@ pipeline that are reproducible offline. What it does and does not cover:
316338
| `rcodesign encode-app-store-connect-api-key` | yes, with `openssl`-generated ECDSA P8 | encode step is purely local, no network |
317339
| `rcodesign notary-submit` (notarization) | no | hardcoded `https://appstoreconnect.apple.com/notary/v2/submissions` endpoint in `app-store-connect/src/notary_api.rs`; no `--dry-run`, no env-var endpoint override |
318340

319-
Reaching Tier 2 (running the indygreg action via `act`) or Tier 3 (mocking
320-
the notary HTTPS endpoint) would require more infrastructure than the
321-
current Tier-1 setup. Tier 1 covers ~80% of the value at ~5% of the
322-
complexity: any divergence between local and CI for the signing/zip/GPG
323-
path will surface; only Apple's notary submission itself is untested
324-
locally.
341+
The one part it cannot cover offline is Apple's notary submission itself
342+
(`rcodesign notary-submit`), which has to reach Apple's servers. Mocking that
343+
HTTPS endpoint would take more infrastructure than the rest of the validator
344+
combined, for the single step a real release exercises anyway. Everything
345+
else, any divergence between local and CI on the sign / zip / GPG path,
346+
surfaces locally.
325347

326348
### Stub Mach-O binaries in the local validator
327349

@@ -368,6 +390,12 @@ revisiting the design:
368390
mounted `OUTPUT_DIR` or the published artifacts
369391
- `Import GPG key` fails fast on an empty `RPM_GPG_PRIVATE_KEY` rather than
370392
letting the no-op signing branch produce an unsigned release
393+
- each signing secret is deleted right after its last consumer, so none
394+
remain on disk once the AWS credentials action and `upload-artifact` run
395+
- each zip's `.sig` is uploaded to S3 before the zip itself, so a partial
396+
upload can never leave a consumable zip without its signature
397+
- the `rcodesign` tarball is verified against a repo-pinned SHA256, not the
398+
upstream `*.sha256` sidecar
371399
- the macOS Mach-O binaries are built natively on `macos-14`; all other
372400
operations run on `ubuntu-24.04`
373401

@@ -376,7 +404,7 @@ revisiting the design:
376404
The current design should be reconsidered if any of these change:
377405

378406
- rcodesign gains a `--dry-run` flag or env-var endpoint override for
379-
notarization (would enable a Tier 3 local mock)
407+
notarization (would let the local validator cover notary submission too)
380408
- Apple migrates away from an rcodesign-compatible API (would force a fall-
381409
back to Apple's `notarytool` or `codesign`, which only run on macOS)
382410
- the security team rotates to a per-format GPG signing key (would require
@@ -404,7 +432,6 @@ The current design should be reconsidered if any of these change:
404432
405433
### Upstream tooling
406434
407-
- [`indygreg/apple-code-sign-action`](https://github.com/indygreg/apple-code-sign-action) - GitHub Action wrapping rcodesign
408435
- [`indygreg/apple-platform-rs`](https://github.com/indygreg/apple-platform-rs) - rcodesign source; relevant code paths: `apple-codesign/src/cli/mod.rs` (CLI surface), `app-store-connect/src/notary_api.rs` (hardcoded notary endpoint)
409436
- [`rcodesign` documentation](https://gregoryszorc.com/docs/apple-codesign/stable/) - including [certificate management](https://gregoryszorc.com/docs/apple-codesign/stable/apple_codesign_certificate_management.html) and [signing reference](https://gregoryszorc.com/docs/apple-codesign/stable/apple_codesign_rcodesign_signing.html)
410-
- [Apple notary service overview](https://developer.apple.com/documentation/security/customizing_the_notarization_workflow)
437+
- [Apple notary service overview](https://developer.apple.com/documentation/security/notarizing-macos-software-before-distribution)

.github/packaging/scripts/build-macos-zip-builder-image.sh

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
# Build the macOS-zip validator Docker image with Go and rcodesign checksum
44
# verification.
55
#
6-
# Fetches the SHA256 checksum for the Go tarball from go.dev release metadata
7-
# and for rcodesign from its release sidecar file, then passes both to the
8-
# Docker build for integrity verification.
6+
# Fetches the SHA256 checksum for the Go tarball from go.dev release metadata,
7+
# uses a repo-pinned digest for rcodesign, then passes both to the Docker build
8+
# for integrity verification.
99
#
1010
# Required env vars:
1111
# GO_VERSION - Go version to install (e.g., "1.24.12")
@@ -24,14 +24,28 @@ set -euo pipefail
2424

2525
command -v jq >/dev/null 2>&1 || { echo "ERROR: jq is required but not found on PATH" >&2; exit 1; }
2626

27-
# Map host arch to Go's and rcodesign's naming conventions
27+
# Map host arch to Go's naming convention, and pin the rcodesign release-tarball
28+
# SHA256 per arch. The digests are pinned in this repo
29+
# (rather than fetched from the release's own .sha256 sidecar) so a retagged or
30+
# replaced upstream release cannot swap the signing binary. Keep in sync with
31+
# APPLE_CODESIGN_SHA256 / APPLE_CODESIGN_PKG_VERSION in
32+
# .github/workflows/build-macos-release.yml. Digests are version-specific.
33+
readonly PINNED_APPLE_CODESIGN_VERSION="0.28.0"
2834
arch=$(uname -m)
2935
case "${arch}" in
30-
x86_64) goarch="amd64"; rcodesign_arch="x86_64-unknown-linux-musl" ;;
31-
aarch64|arm64) goarch="arm64"; rcodesign_arch="aarch64-unknown-linux-musl" ;;
36+
x86_64) goarch="amd64"; rcodesign_checksum="d0f0e4c7915295a1e79fc8b775b354c76d621a52da4c6b841dc9f0585de06cca" ;;
37+
aarch64|arm64) goarch="arm64"; rcodesign_checksum="1d3550680c0444b24b1387753f46efc7dcdb336f9001dbde4508f1c58da90b81" ;;
3238
*) echo "Unsupported arch: ${arch}" >&2; exit 1 ;;
3339
esac
3440

41+
# The pinned digests above are only valid for one rcodesign version. Fail loudly
42+
# if the requested version has drifted, rather than verifying against the wrong
43+
# digest.
44+
if [[ "${APPLE_CODESIGN_VERSION}" != "${PINNED_APPLE_CODESIGN_VERSION}" ]]; then
45+
echo "ERROR: no pinned rcodesign digest for version ${APPLE_CODESIGN_VERSION} (pinned: ${PINNED_APPLE_CODESIGN_VERSION}); update the digests in $(basename "$0")" >&2
46+
exit 1
47+
fi
48+
3549
# ── Go checksum ──────────────────────────────────────────────────────
3650

3751
go_filename="go${GO_VERSION}.linux-${goarch}.tar.gz"
@@ -45,21 +59,7 @@ if [[ -z "${go_checksum}" || "${go_checksum}" == "null" ]]; then
4559
exit 1
4660
fi
4761
echo "Go checksum: ${go_checksum}"
48-
49-
# ── rcodesign checksum ───────────────────────────────────────────────
50-
51-
rcodesign_filename="apple-codesign-${APPLE_CODESIGN_VERSION}-${rcodesign_arch}.tar.gz"
52-
# The tag uses a slash separator (apple-codesign/<ver>) that must be URL-
53-
# encoded as %2F in the download path.
54-
rcodesign_sha_url="https://github.com/indygreg/apple-platform-rs/releases/download/apple-codesign%2F${APPLE_CODESIGN_VERSION}/${rcodesign_filename}.sha256"
55-
echo "Fetching SHA256 checksum for ${rcodesign_filename}..."
56-
rcodesign_checksum=$(curl -fsSL "${rcodesign_sha_url}" | awk '{print $1}')
57-
58-
if [[ -z "${rcodesign_checksum}" ]]; then
59-
echo "ERROR: Could not fetch checksum for ${rcodesign_filename} from ${rcodesign_sha_url}" >&2
60-
exit 1
61-
fi
62-
echo "rcodesign checksum: ${rcodesign_checksum}"
62+
echo "rcodesign checksum (pinned): ${rcodesign_checksum}"
6363

6464
# ── Build ───────────────────────────────────────────────────────────
6565

.github/packaging/scripts/build-macos-zip.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,9 @@ done
108108
# branching (same path RPM/DEB packaging uses). The production script
109109
# (build-zip-pkg.sh) reads GPG_KEY_FILE and GPG_PASSPHRASE.
110110

111+
# setup_gpg (called below) materializes the signing key at NFPM_SIGNING_KEY:
112+
# it writes the freshly-generated ephemeral key there, or copies the provided
113+
# key file to that path. Everything downstream reads the key from this path.
111114
INPUT_GPG_KEY_FILE="${GPG_KEY_FILE:-}"
112115
export NFPM_SIGNING_KEY="${STAGE}/gpg-signing-key.asc"
113116
export GPG_PASSPHRASE="${GPG_KEY_PASSPHRASE:-}"

0 commit comments

Comments
 (0)