Skip to content

Commit 3574f80

Browse files
authored
fix(bundler): cover recipe.yaml with bundle checksums (#1750)
Signed-off-by: Yuan Chen <yuanchen97@gmail.com>
1 parent 86272d0 commit 3574f80

14 files changed

Lines changed: 146 additions & 43 deletions

demos/bundle-attestation-demo-slides.html

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -191,7 +191,7 @@ <h2>Generated artifacts have no upstream tag</h2>
191191
</table>
192192
<div class="grid2">
193193
<div class="mini"><h4>Without attestation</h4><p>Bundle is just files. <code class="inl">checksums.txt</code> catches accidental corruption, nothing else.</p></div>
194-
<div class="mini"><h4>With <code class="inl">--attest</code></h4><p>The files listed in <code class="inl">checksums.txt</code> are cryptographically bound to the creator <em>and</em> the binary that produced it (recipe.yaml excluded, #1549). Recursive provenance.</p></div>
194+
<div class="mini"><h4>With <code class="inl">--attest</code></h4><p>The generated payload files listed in <code class="inl">checksums.txt</code>, including <code class="inl">recipe.yaml</code>, are cryptographically bound to the creator <em>and</em> the binary that produced it. Recursive provenance.</p></div>
195195
</div>
196196
</div>
197197
</section>
@@ -233,7 +233,7 @@ <h2><code class="inl">aicr bundle --attest</code></h2>
233233
<text x="46" y="214">recipe.yaml &#183; deploy.sh &#183; README.md</text>
234234
<text x="46" y="234">&lt;component&gt;/values.yaml &#183; ...</text>
235235
<text x="46" y="266" fill="#9aa7b4">checksums.txt</text>
236-
<text x="46" y="282" fill="#6b7785" font-size="11">(SHA256 of the listed files; recipe.yaml excluded — #1549)</text>
236+
<text x="46" y="282" fill="#6b7785" font-size="11">(SHA256 of every generated payload file)</text>
237237
</g>
238238
<rect class="node-g" x="432" y="170" width="398" height="186" rx="9"/>
239239
<text class="ttl-g nlabel" x="448" y="194">attestation/</text>

demos/bundle-attestation-demo.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -233,7 +233,7 @@ run bash -c "set -o pipefail; '$AICR' verify '$BUNDLE' --format json | jq '{ tru
233233
# --- tamper -------------------------------------------------------------------
234234

235235
banner "Tamper-evident: mutate a content file, verify fails"
236-
note "The signature's subject is the digest of checksums.txt, which pins every file it lists (recipe.yaml is not yet covered, #1549)."
236+
note "The signature's subject is the digest of checksums.txt, which pins every generated payload file, including recipe.yaml."
237237
note "Mutating any file listed in checksums.txt breaks its checksum; mutating checksums.txt breaks the signature."
238238
# Pick a file the bundle is guaranteed to contain. README.md is always present.
239239
TAMPER_TARGET="$BUNDLE/README.md"

demos/bundle-attestation.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,12 @@
33
Bundle attestation provides cryptographic proof of **who** created a deployment
44
bundle and **which AICR CLI** built it. When `aicr bundle --attest` runs, the
55
CLI signs the bundle's `checksums.txt` (which inventories the deployment
6-
payload; `recipe.yaml` is currently excluded, #1549, and the attestation files
7-
are verified separately) using [Sigstore](https://www.sigstore.dev/) and
6+
payload, including `recipe.yaml`; the attestation files are verified separately)
7+
using [Sigstore](https://www.sigstore.dev/) and
88
generates SLSA Build Provenance v1 metadata. Anyone can later verify the
99
bundle with `aicr verify` to confirm:
1010

11-
* The files listed in `checksums.txt` haven't been tampered with (recipe.yaml is excluded, #1549).
11+
* The generated payload files listed in `checksums.txt`, including `recipe.yaml`, haven't been tampered with.
1212
* It was created by a trusted identity.
1313
* It was built by an attested NVIDIA-CI-released AICR CLI.
1414

@@ -88,7 +88,7 @@ permissions.
8888

8989
```text
9090
my-bundle/
91-
├── checksums.txt # SHA256 of every listed file (excludes recipe.yaml, #1549)
91+
├── checksums.txt # SHA256 of every generated payload file
9292
├── recipe.yaml # canonical post-resolution recipe
9393
├── deploy.sh # automation script
9494
├── README.md # deployment guide
@@ -105,10 +105,9 @@ The two attestations together form the chain that makes `verified` reachable:
105105

106106
* **`bundle-attestation.sigstore.json`** — Sigstore Bundle (DSSE + Fulcio cert
107107
+ Rekor inclusion proof). Its in-toto subject is the SHA256 of
108-
`checksums.txt`, so signing this one file transitively pins every content
109-
file that `checksums.txt` lists. The signer identity is the creator's OIDC
110-
identity. (Caveat: `recipe.yaml` is currently written *after* `checksums.txt`
111-
and is not yet covered — tracked in #1549.)
108+
`checksums.txt`, so signing this one file transitively pins every generated
109+
payload file that `checksums.txt` lists, including `recipe.yaml`. The signer
110+
identity is the creator's OIDC identity.
112111
* **`aicr-attestation.sigstore.json`** — the SLSA Build Provenance attestation
113112
*of the AICR CLI binary that produced the bundle*, copied in at bundle time.
114113
Its signer identity is NVIDIA CI (`https://github.com/NVIDIA/aicr/.github/workflows/on-tag.yaml@...`).
@@ -143,7 +142,7 @@ Bundle verification: PASSED
143142

144143
Five gates run, top to bottom:
145144

146-
1. **Checksums** — every file listed in `checksums.txt` is hashed and compared (recipe.yaml is not yet listed — #1549).
145+
1. **Checksums** — every generated payload file listed in `checksums.txt`, including `recipe.yaml`, is hashed and compared.
147146
2. **Bundle signature** — the Sigstore Bundle is verified against the trusted root.
148147
3. **Bundle predicate** — the in-toto subject is checked against the actual `checksums.txt` digest.
149148
4. **Binary attestation chain**`aicr-attestation.sigstore.json` is verified and its subject is checked against the CLI binary digest claimed in the bundle predicate.

demos/cuj2-demo.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,8 +37,8 @@
3737
│ recipe.yaml ──▶ bundle/ │
3838
│ ├── deploy.sh (root automation script) │
3939
│ ├── README.md (root deployment guide) │
40-
│ ├── checksums.txt (SHA256 of listed files; excludes recipe.yaml)
41-
│ ├── recipe.yaml (resolved recipe; not yet in checksums, #1549)
40+
│ ├── checksums.txt (SHA256 of all generated payload files)
41+
│ ├── recipe.yaml (resolved recipe; covered by checksums)
4242
│ ├── 001-agentgateway-crds/ (agentgateway.dev CRDs) │
4343
│ ├── 002-agentgateway-crds-post/ (Gateway API + Inf-Ext CRDs)│
4444
│ ├── 003-aws-ebs-csi-driver/ (EBS storage) │

demos/images/data.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,7 @@ Visual: File system tree structure showing multiple component folders
4242
bundle/
4343
├── deploy.sh # root automation script (executable)
4444
├── README.md # root deployment guide
45-
├── checksums.txt # SHA256 of the listed bundle files (recipe.yaml is written
46-
# afterward and is not yet covered — see issue #1549)
45+
├── checksums.txt # SHA256 of every generated payload file
4746
├── recipe.yaml # canonical post-resolution recipe
4847
├── NNN-cert-manager/ # each folder is prefixed with its NNN deployment-order
4948
│ ├── install.sh # index (computed from the dependency graph — e.g. cert-manager

docs/integrator/data-flow.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -385,8 +385,8 @@ RecipeResult
385385
CR, for components that ship one)
386386
- go:embed templates -> per-component install.sh, and the root
387387
README.md + deploy.sh
388+
-> write canonical recipe.yaml
388389
-> compute root checksums.txt over every emitted file
389-
(recipe.yaml is written afterward and is not covered, #1549)
390390
```
391391

392392
`pkg/bundler/registry` exists but is **not** used by the production path: the
@@ -507,7 +507,8 @@ The `deploymentOrder` field in recipes specifies component deployment sequence.
507507
bundle-output/
508508
├── README.md # Root deployment guide with ordered steps
509509
├── deploy.sh # Automation script (0755)
510-
├── checksums.txt # SHA256 of all listed files (recipe.yaml excluded, #1549)
510+
├── recipe.yaml # Canonical post-resolution recipe
511+
├── checksums.txt # SHA256 of all generated payload files
511512
├── 001-cert-manager/
512513
│ ├── install.sh # Per-folder install script (0755)
513514
│ ├── values.yaml # Static Helm values
@@ -584,7 +585,7 @@ spec:
584585
│ Complete Bundle + Deploy Flow │
585586
├──────────────────────────────────────────────────────────────┤
586587
│ │
587-
│ aicr bundle -r recipe.yaml --deployer argocd \ │
588+
│ aicr bundle -r recipe.yaml --deployer argocd \
588589
│ --repo https://github.com/my-org/my-repo.git -o ./out │
589590
│ │
590591
│ 1. Parse recipe │
@@ -599,13 +600,13 @@ spec:
599600
│ └─ network-operator → values.yaml, manifests/ │
600601
│ │
601602
│ 4. Run deployer (argocd) → numbered NNN-<name>/ folders │
602-
│ ├─ 001-cert-manager/application.yaml (wave: 0) │
603-
│ ├─ 002-gpu-operator/application.yaml (wave: 1) │
604-
│ └─ 003-network-operator/application.yaml (wave: 2) │
603+
│ ├─ 001-cert-manager/application.yaml (wave: 0)
604+
│ ├─ 002-gpu-operator/application.yaml (wave: 1)
605+
│ └─ 003-network-operator/application.yaml (wave: 2)
605606
│ └─ app-of-apps.yaml (bundle root, uses --repo URL) │
606607
│ │
607608
│ 5. Generate checksums │
608-
│ └─ checksums.txt (root; all listed files, no recipe.yaml)
609+
│ └─ checksums.txt (root; all emitted Argo CD files)
609610
│ │
610611
└──────────────────────────────────────────────────────────────┘
611612
```

docs/integrator/openshift.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ ocp-bundle/
146146
├── undeploy.sh # Cleanup script
147147
├── README.md # Bundle documentation
148148
├── recipe.yaml # Recipe used to generate bundle
149-
├── checksums.txt # SHA256 of all listed files (recipe.yaml excluded, #1549)
149+
├── checksums.txt # SHA256 of all generated payload files, including recipe.yaml
150150
151151
│ # ── Per-operator three-folder cycle ──
152152

docs/integrator/supply-chain-verification.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -305,8 +305,8 @@ release.
305305
### Bundle attestation
306306

307307
When `aicr bundle` runs with `--attest`, it signs the bundle using Sigstore
308-
keyless OIDC, binding the bundle creator's identity to the files listed in `checksums.txt`
309-
(recipe.yaml is currently excluded, #1549) and the binary that produced it (via
308+
keyless OIDC, binding the bundle creator's identity to the generated payload
309+
files listed in `checksums.txt`, including `recipe.yaml` in Helm bundles, and the binary that produced it (via
310310
`resolvedDependencies`). Attestation is opt-in; bundles are unsigned by
311311
default. The bundle output includes `bundle-attestation.sigstore.json`
312312
(SLSA Build Provenance v1 for the bundle) and a copy of the binary's

docs/user/artifact-verification.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ aicr verify ./my-bundle
5353

5454
Under the hood this runs three checks:
5555

56-
1. **Checksums**: every file listed in `checksums.txt` is hashed and matched (recipe.yaml is not yet listed — #1549).
56+
1. **Checksums**: every generated payload file listed in `checksums.txt`, including `recipe.yaml` in Helm bundles, is hashed and matched.
5757
2. **Bundle attestation**: the bundle's signature is verified against the Sigstore trusted root.
5858
3. **Binary attestation**: the provenance chain is verified with identity pinned to NVIDIA CI.
5959

docs/user/cli-reference.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1747,7 +1747,7 @@ Use `--dynamic` for values that genuinely vary per cluster — cluster names, su
17471747
| Cluster-specific value (varies per deployment) | `--dynamic` | `--dynamic alloy:clusterName` |
17481748
| Static override (same for all deployments of this bundle) | `--set` | `--set gpuoperator:driver.version=580.105.08` |
17491749

1750-
> **Attestation scope:** Dynamic values are supplied at install time and are **not covered by `--attest`**. Attestation binds the files listed in `checksums.txt` (recipe.yaml excluded, #1549), not operator-provided overrides. If you need to constrain dynamic values at deploy time, use admission control or Argo sync hooks — see [Attestation Scope](#attestation-scope).
1750+
> **Attestation scope:** Dynamic values are supplied at install time and are **not covered by `--attest`**. Attestation binds the generated payload files listed in `checksums.txt`, including `recipe.yaml` in Helm bundles, not operator-provided overrides. If you need to constrain dynamic values at deploy time, use admission control or Argo sync hooks — see [Attestation Scope](#attestation-scope).
17511751

17521752
```shell
17531753
--dynamic component:path.to.field
@@ -2169,7 +2169,7 @@ When `--attest` is passed, the bundle command performs five steps:
21692169
1. **Verifies the binary attestation file exists** — The running `aicr` binary must have a valid SLSA provenance file (`aicr-attestation.sigstore.json`) alongside it, included by the install script from a release archive. If missing, the command fails immediately with guidance on how to install correctly.
21702170
2. **Acquires a signing credential** — in the default keyless mode this is an OIDC token (see [OIDC Token Sources](#oidc-token-sources) below); with `--signing-key` this step instead resolves the KMS key and no OIDC token is acquired (see [KMS-Backed Signing](#kms-backed-signing)).
21712171
3. **Verifies the binary's own attestation** — Cryptographically verifies the SLSA provenance binds to the running binary and was signed by NVIDIA CI. This ensures only NVIDIA-built binaries can produce attested bundles.
2172-
4. **Signs the bundle** — Creates a SLSA Build Provenance v1 in-toto statement binding the creator's identity to the files listed in `checksums.txt` (recipe.yaml is currently excluded, #1549) and the binary that produced it.
2172+
4. **Signs the bundle** — Creates a SLSA Build Provenance v1 in-toto statement binding the creator's identity to the generated payload files listed in `checksums.txt`, including `recipe.yaml` in Helm bundles, and the binary that produced it.
21732173
5. **Writes attestation files**`attestation/bundle-attestation.sigstore.json` and `attestation/aicr-attestation.sigstore.json` are added to the bundle output.
21742174

21752175
Attestation is opt-in; bundles are unsigned by default. By default, signing uses Sigstore keyless signing (Fulcio CA + Rekor transparency log) and records the entry in **Rekor v2** (the signing config is fetched from Sigstore's TUF repository, so shard rotation is handled automatically; a cold cache is fetched on demand). Verifying such bundles with `aicr verify` needs only the `aicr` binary; verifying them with `cosign verify-blob-attestation` needs Cosign v3.0.1+. For CI/CD environments without OIDC, pass `--signing-key` to sign with a cloud KMS key instead; see [KMS-Backed Signing](#kms-backed-signing) below. For verification, see [`aicr verify`](#aicr-verify).
@@ -2296,7 +2296,7 @@ HashiCorp Vault (`hashivault://`) is not supported: its client libraries are MPL
22962296

22972297
##### Attestation Scope
22982298

2299-
Attestation binds the files listed in `checksums.txt` (recipe.yaml is currently excluded, #1549; the attestation files are verified separately) — defaults, dynamic-value stubs, and any external `--data` files copied into the bundle. It does **not** bind install-time values supplied via `helm --set`, a user-provided `-f extra.yaml`, or Argo `Application.spec.source.helm.parameters`. That boundary is intentional: dynamic values are the operator's domain by design.
2299+
Attestation binds the generated payload files listed in `checksums.txt` — including `recipe.yaml` in Helm bundles, defaults, dynamic-value stubs, and any external `--data` files copied into the bundle. The attestation files are verified separately. It does **not** bind install-time values supplied via `helm --set`, a user-provided `-f extra.yaml`, or Argo `Application.spec.source.helm.parameters`. That boundary is intentional: dynamic values are the operator's domain by design.
23002300

23012301
If you need to enforce specific install-time values (e.g., pinning `driver.version`), that is a **policy concern**, not an attestation one. Use admission control (Kyverno, Gatekeeper) or Argo sync hooks to reject deployments that violate the policy. `aicr verify` checks bundle integrity and provenance; it does not evaluate install-time value constraints.
23022302

0 commit comments

Comments
 (0)