@@ -429,87 +429,40 @@ certificate** — the onboarded KMS re-issues its own cert, so the PEM differs
429429while ` openssl x509 -pubkey ` is identical. A KMS that bootstrapped independently
430430differs in both.
431431
432- ### 9.4 Provision host certificates first
433-
434- Verification needs the ASK and VCEK certificates. By default they are fetched
435- from ` https://kdsintf.amd.com ` , which is a ** single global endpoint with no
436- mirror** , is aggressively rate-limited (one identical request per ~ 10s), and was
437- completely unreachable for an entire test round — TCP timeouts and 100% ICMP loss
438- from five vantage points across four autonomous systems and two continents, while
439- every other external service answered normally. DNS was healthy and unchanged
440- throughout, so this was the service, not a migration.
441-
442- The rate limit is not theoretical: back-to-back negative tests against the same
443- chip hit ` HTTP status client error (429) ` on the VCEK URL, which reads exactly
444- like a verification failure until you notice the status code. Space repeat
445- verifications ~ 25s apart, or provision the host.
446-
447- dstack already supports the alternative: ` normalize_kernel_cert_table ` reads ASK
448- and VCEK from the SNP extended report's certificate table, so a host that has
449- been provisioned needs no KDS access at all. AMD's own specification recommends
450- caching at the host for exactly this reason.
451-
452- ** There is no KDS cache service in this repo, and nothing to point one at.** The
453- only caching is in-process: ` AmdKdsClient ` holds a ` moka ` cache of 16 CA chains
454- and 1024 VCEKs, bounded by capacity with no TTL and no persistence, so it is warm
455- only for the lifetime of a single KMS or verifier process. A one-shot
456- ` dstack-verifier --verify ` starts cold every time, which is exactly how the 429
457- above was earned. ` [core.attestation.urls] amd_kds ` accepts any KDS-compatible
458- base URL, so a mirror would drop in — but you would have to write it. Compare
459- ` nvidia-attest-proxy ` , which is the persistent, on-disk, PCCS-like cache the
460- NVIDIA collateral got and AMD did not. (` crates/mock-attestation ` serves
461- KDS-shaped routes but is a test fixture with fake collateral, not a mirror.)
462-
463- The guest side of the cert-table path needs nothing configured: ` sev-snp-attest `
464- reads ` certs ` , ` cert_chain ` or ` auxblob ` from the configfs TSM report, falling
465- back to the extended-report ioctl, and passes whatever it finds up with the
466- quote. An unprovisioned host simply yields an empty chain, and the verifier then
467- goes to KDS — which is what happened here.
468-
469- ** Check that your VMM can actually deliver those certificates before planning
470- around it.** On the host used for this round it cannot, and the earlier revision
471- of this section recommended provisioning without checking.
472-
473- The tooling half is fine. ` snphost ` (virtee) is the SEV-SNP host CLI:
474- ` snphost fetch ca [der|pem] DIR ` and ` snphost fetch vek [der|pem] DIR ` pull the
475- CA chain and the VCEK from KDS, and ` snphost import DIR CERT-FILE ` packs them
476- into the GHCB-formatted blob that, in its own words, "can then be provided to
477- QEMU to perform extended attestation on guests". (` sevctl ` is * not* part of this
478- — it manages the pre-SNP SEV platform and its OCA certificate chain. A previous
479- revision of this section listed the two together; that was wrong.)
480-
481- The delivery half is the problem. Providing that blob to QEMU needs a
482- ` certs-path ` -style property on the ` sev-snp-guest ` object, and stock QEMU does
483- not have one:
484-
485- ```
486- $ qemu-system-x86_64 -object sev-snp-guest,help # QEMU 10.2.1
487- sev-snp-guest options:
488- author-key-enabled=<bool> guest-visible-workarounds=<string>
489- host-data=<string> id-auth=<string>
490- id-block=<string> kernel-hashes=<bool>
491- policy=<uint64> sev-device=<string>
492- vcek-disabled=<bool>
493- ```
494-
495- ` strings ` on the binary confirms it — no ` certs-path ` anywhere. The host-side
496- ` SNP_SET_EXT_CONFIG ` ioctl, the other way certificates used to be staged, is
497- absent from upstream ` linux/psp-sev.h ` as well; it lived in AMD's out-of-tree
498- fork. The guest half is present and working — ` linux/sev-guest.h ` still has
499- ` SNP_GET_EXT_REPORT ` and its ` certs_address ` — so the guest asks and gets an
500- empty table back.
501-
502- Which is exactly what was observed: ` sev-snp-attest ` found no ` certs ` ,
503- ` cert_chain ` or ` auxblob ` content, so the attestation travelled with an empty
504- chain and the verifier went to KDS every time.
505-
506- ** So, on a stock Ubuntu/Debian QEMU, KDS is not optional.** The realistic
507- mitigations are the boring ones: keep a long-lived KMS so its in-process cache
508- stays warm, and space repeat verifications out. Removing the dependency properly
509- means either a QEMU that can carry the cert blob, or a KDS mirror behind
510- ` [core.attestation.urls] amd_kds ` . Both are work, not configuration. And note
511- the bootstrap problem either way — obtaining the VCEK requires KDS access once,
512- from a network that can reach it.
432+ ### 9.4 KDS is on the critical path, and stays there
433+
434+ Verification needs the ASK and VCEK certificates. They come from
435+ ` https://kdsintf.amd.com ` , a ** single global endpoint with no mirror** , rate
436+ limited to roughly one identical request per 10s, and unreachable for an entire
437+ test round — TCP timeouts and 100% ICMP loss from five vantage points across four
438+ autonomous systems and two continents, while every other external service
439+ answered normally and DNS stayed healthy throughout.
440+
441+ The rate limit is the one that will bite you mid-test: back-to-back verifications
442+ of the same chip return ` HTTP status client error (429) ` on the VCEK URL, which
443+ reads exactly like a verification failure until you notice the status code. Space
444+ them ~ 25s apart.
445+
446+ Caching is in-process only. ` AmdKdsClient ` keeps a ` moka ` cache of 16 CA chains
447+ and 1024 VCEKs, capacity-bounded, no TTL, no persistence — warm for the life of
448+ one KMS or verifier process and cold in every one-shot ` dstack-verifier --verify ` .
449+ There is no KDS cache service in this repo; ` [core.attestation.urls] amd_kds `
450+ would accept a mirror, but nothing implements one. (Compare ` nvidia-attest-proxy ` ,
451+ the persistent on-disk cache the NVIDIA collateral got and AMD did not.)
452+
453+ Do not plan on serving the certificates from the host instead. The verifier side
454+ supports it — ` normalize_kernel_cert_table ` reads ASK and VCEK from the SNP
455+ extended report's certificate table — and so does the guest, but nothing can put
456+ them there: ` sev-snp-guest ` in stock QEMU has no ` certs-path ` -style property, and
457+ the host-side ` SNP_SET_EXT_CONFIG ` ioctl is not in upstream ` psp-sev.h ` . Guests
458+ ask via ` SNP_GET_EXT_REPORT ` and get an empty table, which is why every
459+ verification in this round went to KDS.
460+
461+ A note for whoever touches this code: the ARK is already compiled in, and the ARK
462+ fetched from KDS is discarded (` let (_fetched_ark, ask) = ... ` ). The network
463+ request exists only to obtain the ASK, which is equally static per product family.
464+ Bundling it would remove the ` cert_chain ` request entirely, leaving only the
465+ per-chip VCEK.
513466
514467### 9.5 Reaching the API that holds the keys
515468
@@ -539,12 +492,6 @@ these JSON bodies are **hex strings**, not base64 and not byte arrays. Base64
539492fails with `Invalid character 'w' at position 5`, which names the symptom and not
540493the cause.
541494
542- A related note for whoever touches this code : the ARK is already compiled in, and
543- the ARK fetched from KDS is discarded (`let (_fetched_ark, ask) = ...`). The
544- network request exists only to obtain the ASK, which is equally static per product
545- family. Bundling it would remove the `cert_chain` request entirely, leaving only
546- the per-chip VCEK.
547-
548495---
549496
550497# # 10. Fixed in 0.6.0-rc0 testing — do not re-diagnose
0 commit comments