vtpm: RFC — persistent vTPM state via a pluggable external TPM endpoint#1091
vtpm: RFC — persistent vTPM state via a pluggable external TPM endpoint#1091Goodleon-Y wants to merge 8 commits into
Conversation
Adds a small C-level accessor layer (state_accessor.{c,h}) that exposes
the TPM 2.0 Reference Implementation's persistent globals (gp, gc, gr)
via stable getter/setter functions plus a bulk serialize/deserialize
pair. The libtcgtpm Makefile is extended to compile state_accessor.c
into a separate libStateAccessor.a archive, and build.rs is updated to
link it alongside libcrt/libcrypto/libtcgtpm.
This is the foundation for any out-of-CVM TPM state persistence path
(state dump/restore, sealed-blob snapshotting, or KBS-stateful vTPM).
Signed-off-by: Goodleon-Y <goodleon3@126.com>
Introduces TpmTransport — a small trait abstracting any TPM 2.0
command/response endpoint outside the CVM (software vTPM, physical
discrete TPM, KBS-backed TPM service, ...). Adds:
* TpmProxy<T: TpmTransport> — constructs raw TPM 2.0 command
buffers (CreatePrimary, Create, Load, Unseal, FlushContext, NV_*)
and dispatches them over the supplied transport.
* MockTransport — records sent commands and returns
canned responses; used by unit tests of TpmProxy logic without
needing a real TPM.
The transport boundary is deliberately backend-agnostic to align with
the upcoming "Move vTPM to user-mode" effort and any future block-layer
or KBS-stateful TPM persistence approach.
Signed-off-by: Goodleon-Y <goodleon3@126.com>
Implements TpmTransport over AF_VSOCK using SVSM's existing VsockStream. Each TPM command opens a fresh connection, sends the command frame, reads the response header (10 bytes: tag(2)+size(4)+rc(4)) to learn the total length, then reads the body. One-command-per- connection avoids carrying TPM session state across the boundary. Gated behind the existing 'vsock' feature. Default builds are unaffected. The host side needs a lightweight forwarder, e.g.: socat VSOCK-LISTEN:9999,fork OPEN:/dev/tpm0 Signed-off-by: Goodleon-Y <goodleon3@126.com>
Introduces SealedBlobStore — a backend-agnostic trait for persisting
the sealed vTPM state blob across cold boots. Three bundled
implementations:
* StaticBufStore — in-CVM static buffer. Survives warm reboots
within a single CVM lifetime; zeroed on cold
boot. Suitable for development and bring-up.
* IgvmVarStore — stub for an IGVM variable-backed store
(production target; load/save plumbing TBD).
* VsockHostStore — AF_VSOCK to a host-side persistence helper
(feature-gated 'vsock'). Two ports separate
the load and save directions so the helper can
implement them as independent services.
The trait is intentionally backend-agnostic to align with the upcoming
COCONUT-SVSM block-layer abstraction and any KBS-stateful TPM service.
Signed-off-by: Goodleon-Y <goodleon3@126.com>
Adds VtpmState (serializable view of the persistent TPM state needed
across cold boots) and SealedBlob — a packed binary container
combining two encryption layers:
Layer 1: AES-256-GCM over the serialized VtpmState. Key + nonce are
derived from a caller-supplied entropy bundle (the kernel
init path uses dual-source HKDF).
Layer 2: TPM2_Seal of a small (60-byte) key bundle holding the AES
key, GCM tag and nonce. This keeps the TPM-sealed payload
well below RSA-2048 size limits while still binding the
bulk data to a TPM-resident parent key.
The packed container is self-describing (magic + version + counter +
PCR digest + length-prefixed sections + SHA-256 trailer) so it can be
written through SealedBlobStore (commit 4) and read back unchanged.
Anti-rollback support: an HMAC-bound monotonic counter is part of the
header so storage backends that share a counter across tenants
(NV index pool) can still distinguish per-VM blob versions.
Signed-off-by: Goodleon-Y <goodleon3@126.com>
Adds VtpmInternalState plus extract_vtpm_state() / inject_vtpm_state()
that call into the C-side accessors exposed by libtcgtpm
(state_accessor.{c,h}, commit 1) to read and write the TPM 2.0
Reference Implementation's persistent globals (gp, gc, gr).
A bulk-serialization path is also wired up:
* extract_serialized_state() / inject_serialized_state() round-trip
the entire set of accessor fields as one opaque buffer, which is
what the seal/unseal flow stores inside a SealedBlob.
* A command-based fallback (extract_vtpm_state_via_commands) is
sketched for environments where the direct accessors are
unavailable.
This is the bridge that lets the Rust persistence layer recover a
manufactured vTPM byte-for-byte after a cold boot.
Signed-off-by: Goodleon-Y <goodleon3@126.com>
Wires the seal/unseal lifecycle into kernel init using the previously
added components:
* Adds VtpmBootMode (Provision/Recover) and vtpm_init_sealed<T>
in vtpm/mod.rs. Provision manufactures the vTPM, extracts the
serialized internal state, AES-256-GCMs it, TPM2_Seals the 60-byte
key bundle and returns a SealedBlob ready for storage. Recover
reverses the operation: unpack -> TPM2_Unseal -> AES decrypt ->
inject_serialized_state -> light power-on.
* Replaces the previous `static mut SEALED_BLOB_BUF: [u8; 4096]`
scaffolding in svsm.rs with a `&dyn SealedBlobStore` wired through
StaticBufStore. Whether the blob comes from a static buffer (test),
an IGVM variable (production), or a vsock-backed host helper is now
a backend selection rather than a code change.
Entropy comes from RDSEED inside the CVM; key material is zeroized
after the sealing step via volatile writes.
The boot integration is intentionally kept narrow so that a future
user-mode vTPM placement (per the upstream "Move vTPM to user-mode"
plan) can reuse the same persistence trait by selecting a different
SealedBlobStore impl.
Signed-off-by: Goodleon-Y <goodleon3@126.com>
Adds a runtime re-seal path that fires on a successful guest
TPM2_Shutdown (cmdCode 0x145, rc=0) and the supporting Tier-2
NV-block extraction needed to round-trip the simulator's persistent
state across a cold boot.
* libtcgtpm: expose get_nv_blob() / apply_nv_blob() accessors so
the in-process TPM Reference Implementation's 16 KiB NV image
(STATE_NV_BACKUP_SIZE) can be captured at re-seal time and
replayed on Recover. The bulk serialization wrapper grows a new
tagged section (0x12) that carries the NV blob alongside the
existing gp/gc/gr globals.
* vtpm::reseal: new module hosting trigger_if_shutdown() and
do_reseal(). The hook is invoked from TcgTpm::send_tpm_command
*outside* the VTPM SpinLock — re-seal talks to the TPM endpoint
through a separate TpmProxy<VsockTransport> and reads internal
state through the libtcgtpm C FFI, so it never re-acquires VTPM.
Re-seal failures are logged but never propagated to the guest;
fail-closed enforcement remains on the next cold-boot unseal
path (NV-counter check + AES-GCM tag verify).
* vtpm::mod / svsm: wire the optional vtpm-persist feature so the
TPM endpoint and host blob store can be reached over VSOCK
during cold-boot Provision/Recover and runtime re-seal. The
persistence path is opt-in and never silently falls back to
ephemeral — an unreachable endpoint is a hard error so the
security model that treats the physical TPM as a separate root
of trust cannot be downgraded by a channel-only attacker.
* cpu::apic / sev::snp_apic: convert the previously panicking
icr_write() to a logged warning so a transient HV write error
on the persistence path does not crash the boot CPU.
Signed-off-by: Goodleon-Y <goodleon3@126.com>
Goodleon-Y
force-pushed
the
vtpm/tpm-sealing
branch
from
b1fbbd1 to
9aa0f28
Compare
|
Thanks for your submission. Reading through the description above I have a couple of questions around security properties of the host-side parts and actual use-cases for these changes. Can you please join one of the next community calls to discuss these open questions? |
Hi Joerg, thanks for going through this. On the call: I'd rather answer in writing here first, so the technical details stay on the list and are easy to reference, and I can be more precise that way. Glad to join a later call if that's still helpful. On the host-side security. The host components (the VSOCK helper that stores the blob and the bridge to the external TPM) are untrusted, at the same level as the rest of the SEV-SNP host. What the guest relies on today is confidentiality and integrity of the state at rest: the payload is AES-256-GCM encrypted and the AES key is TPM2_Sealed to the external endpoint, so the host only ever handles ciphertext plus a sealed key blob it cannot open. A host that flips bytes in either part fails the GCM tag or the unseal, so silent tampering is caught. What a host can still do is withhold the blob or hand back an older one. As I flagged in open questions 5 and 8, rollback/replay resistance and channel authentication are the parts I left sketched rather than built in this RFC. The shape I'm aiming for is to bind a monotonic counter from the endpoint's NV into the blob and check it on unseal, so a stale or rolled-back blob fails closed, and to wrap the proxy channel in a salted HMAC session so the host can't splice or forge proxy traffic. I'd be glad to turn those two from a sketch into a concrete design note here if that's the most useful next step. With them in place the host reduces to untrusted storage and transport whose only remaining power is denial of service. On the use-case, to be more concrete than the list above: the target is an on-prem or edge confidential VM that needs a stable TPM-rooted identity across reboots (its EK/AK, sealed secrets, measured-boot NV) in a setting where there is no KBS and no guarantee the network is up at boot. Today such a VM either loses vTPM state on every cold boot or has to re-enroll out of band. If the platform already has a discrete TPM, this lets that TPM be the persistence anchor without standing up KBS infrastructure. It is meant to sit beside the encrypted-block + KBS path, not replace it. Happy to go deeper on either. Thanks again. |
|
@Goodleon-Y thanks for this RFC, the design is interesting, but I have a question about the threat model, specifically around the trust boundary between the CVM and the TPM endpoint. The core issue is that the TPM endpoint (whether swtpm or a physical TPM) is reachable only through the host. The host mediates every TPM command and response, and the AES-256 key is exposed in plaintext on the VSOCK channel in both directions during the seal (this may be not a problem if the sealing is done in a trusted environment) but also during the unseal IIUC. This is not just a VSOCK problem: encrypting the VSOCK channel (e.g. with TLS) would not help, because the host-side process that terminates the channel (the socat/bridge to Could you clarify the intended threat model?
For comparison, in the KBS-based approach the CVM generates a key pair, includes the public key in the SNP attestation report (which is signed by the hardware), and the KBS encrypts the secret to that public key after verifying the attestation. The host relays the encrypted blob but never sees the plaintext key, because only the CVM holds the private key. I think understanding the intended trust boundary would help evaluate where this approach fits and what security guarantees it actually provides. As Joerg suggested, I think joining one of the community calls would be really valuable to discuss these points. It would also be a good chance to get broader input from other contributors. |
|
@stefano-garzarella, thanks for the careful read, this is exactly the boundary that matters. You're right, in the RFC the AES key is sealed to the TPM, so on unseal it comes back through the host bridge in cleartext, which makes the host trusted for confidentiality. I've reworked the key schedule so it isn't. The AES key is now derived inside the CVM from two independent halves, and is never sealed or put on the channel:
So sniffing VSOCK, reading the bridge process, or unsealing the TPM blob all yield only The host can't derive Validated on real SEV-SNP with a discrete TPM (Infineon SLB 9672), not swtpm: The known cost of binding to MEASUREMENT is that an SVSM/TCB upgrade changes So it stays complementary to the encrypted-block + KBS path, for deployments with no online broker. Happy to take this to a call, and to share the reworked design or branch if that's useful. |
|
@Goodleon-Y But if Could we just encrypt the vTPM state with the PSP-derived key and store it anywhere, without involving an external TPM at all? |
|
@stefano-garzarella For confidentiality against the host, Rollback: SNP derivation gives you a key but no host-resettable monotonic state, so encrypting with the PSP key and storing the blob anywhere doesn't stop the host withholding it or serving an older one. The NV counter does: we bind its value into the blob and require strict equality on unseal, so a stale or rolled-back blob fails closed. That's why it's a TPM and not just a file. Second root: So encrypt-with-PSP-and-store is valid and simpler if you accept being rollback-able and resting the long-term identity's confidentiality on the single PSP root. The dTPM removes both, with hardware on-prem and edge boxes already have; its cost (MEASUREMENT binding, so upgrades need a migration) I noted before. |
Sorry, the TPM is host-controlled, no? So how can you trust the NV counter? As you mention the TPM can be a software TPM emulated by the host, so the host can do whatever, no?
Okay, so you must trust the TPM attached to the host. How? |
3 participants
[Draft PR — RFC for design feedback, not requesting merge.]
Background
SEV-SNP does not currently provide trusted access from a CVM to
platform devices, including a physical TPM (discrete, firmware, or
otherwise). As a result, the in-CVM vTPM at VMPL0 cannot use a
physical TPM as a backing root of trust and cannot cross a cold boot
on its own. Today, the upstream path being developed by Stefano
Garzarella and others addresses this with encrypted-block + KBS:
durable storage for the vTPM blob plus a remote attestation-gated
key release for decryption. That path fits cloud deployments where
a KBS is already part of the trust infrastructure.
This RFC explores a complementary path for deployments where an
online KBS is unavailable, not a fit for the deployment model, or
overkill for the workload:
trust
disproportionate to the task
The approach: keep the vTPM in-CVM at VMPL0 (no change to today's
attestation surface), but route only its persistent state through a
seal/unseal cycle against an external TPM 2.0 endpoint reached
over an adversarial host channel. The endpoint is intentionally
abstract: a TPM 2.0 command/response pair is all the proxy assumes,
so the same code path can target:
swtpm) for development and CIexists on the platform
persistence anchor maintained outside the CVM
key custody without a full KBS stack
The choice is a deployment decision; the SVSM code is the same.
This is not a replacement for the KBS-stateful vTPM work. The two
approaches target different deployment regimes and can coexist on the
same SVSM build via different backends behind the
SealedBlobStoretrait introduced here.
What this RFC adds
d62abbb): getter/setter accessors for TPM 2.0reference globals (
gp,gc,gr) plus a bulk serialize/deserializepair. Useful for any path that needs to lift the reference
implementation's persistent state out of the in-CVM binary, not just
this RFC.
48abdc3,6c08cd2): a smallTpmTransporttrait with a
MockTransportfor unit tests and a VSOCKimplementation (feature-gated). The TPM2 command builder
(
build_get_random,build_nv_define_space,build_nv_increment,build_nv_read,build_create_primary,build_create,build_load,build_seal,build_unseal,build_flush_context) is generic overthe transport, so the proxy makes no assumption about which TPM 2.0
endpoint is on the other side.
0bdc9df): aSealedBlobStoretrait withStaticBufStore(in-memory, test-only), a stubbedIgvmVarStore,and a VSOCK-backed
VsockHostStore(feature-gated). Intent is toalign with the upcoming block-layer abstraction so the same trait
can later wrap a block device, and to leave room for a KBS-backed
implementation to drop in as a peer backend.
72f12ef): two-layer container.AES-256-GCM-encrypt the state payload (~1 KB at Tier-1.5, growing
to ~19 KB once Tier-2 NV-block persistence is enabled), then
TPM2_Seal only the 60-byte key bundle
(
aes_key[32] || gcm_tag[16] || nonce[12]). Keeps the TPM-sealedpayload well under RSA-2048 size limits regardless of the
underlying state size. The blob format is documented in
kernel/src/vtpm/sealed.rsand is versioned. The current revisioncarries only the AES-256-GCM payload and the TPM-sealed key bundle;
a counter-binding field for cross-tenant replay isolation on a
shared NV counter backend is a planned format addition, not part of
this revision (see open questions 5 and 8).
143ab03): Rust wrappers around the newlibtcgtpm accessors, plus a bulk serialize/deserialize path that
matches the C-side layout.
6f5db7b): two boot modes plumbedthrough
kernel/src/svsm.rs:Provision— manufacture vTPM, extract state, seal via theconfigured TPM endpoint, persist the blob.
Recover— load the blob, unseal via the configured TPM endpoint,inject state, signal NV-on + light power-on.
b1fbbd1): fullguest-state persistence path covering DA lockout, PCR allocation,
orderly-shutdown state, and the platform NV memory (TPM reference
implementation's flat 16 KB
s_NV[]buffer that backs every user NVindex, evict object, and DA bookkeeping field). The libtcgtpm
accessors gain section 0x1A (s_NV dump); the SVSM kernel gains
kernel/src/vtpm/reseal.rsregistering a runtime hook in thecommand path that, on a successful guest-side
TPM2_Shutdown(SU_STATE), re-extracts internal state, freshlyre-encrypts under a new AES key + nonce (no key material is cached
across calls), and pushes a new SealedBlob via
VsockHostStore.Sealed blob grows from ~2.7 KB (Tier-1.5) to ~19 KB (Tier-2). The
hook is feature-gated behind
vtpm-persist; the previous Tier-1.5build path is unchanged when the feature is off.
Known limitations / open questions
StaticBufStore(test-only) andstubbed
IgvmVarStore. Production needs either an IGVM-variablepath, integration with the upcoming SVSM block layer, or a
KBS-backed implementation. I deferred picking one so the trait
shape can be reviewed first.
IGVM
guest_contextonce that surface stabilizes.TpmTransporttrait is intentionally minimal so alternativetransports (virtio-serial, MMIO, in-process for
swtpmco-tenants,etc.) are welcome.
globals via the libtcgtpm accessors added in commit 1. A
TPM2_ContextSave-based path was considered but cannot reach
hierarchy seeds and auth values by design. Would appreciate
guidance on whether the maintainers prefer the direct-accessor
approach or want a higher-level abstraction.
host channel, but per-session HMAC binding is only sketched.
Enforcement would need a session-key provisioning path, which I
left out of this RFC to keep the diff small.
DEVELOPMENT-PLAN.md, the vTPM isslated to move to a user-mode task. The boot integration in commit
7 is intentionally a thin shim (
vtpm_init_sealed) so it can bere-targeted to a user-mode init task. Happy to refactor if there
is a preferred timeline for that move.
TPM2_Shutdown(SU_STATE)only. A guest that does not issueShutdown (forced poweroff, hypervisor-side kill, guest crash,
sudden power loss) loses that session's NV writes. This matches
TCG semantics for ungraceful power loss but is worth flagging as
an API contract.
resistance is sketched, not wired in this revision. The intended
shape is to bind a monotonic counter from the external endpoint's
NV into the blob and check it on unseal, so a stale or rolled-back
blob fails closed. With that in place, the ordering matters: if the
NV counter is advanced inside
seal_statebefore the new SealedBlobis committed via the configured
SealedBlobStorebackend and thestore-side write then fails, a later cold boot would see a counter
ahead of the persisted blob and fall back to Provision instead of
recovering. A save-then-commit ordering (defer
nv_incrementuntilthe store has acknowledged the new blob) avoids that window. This is
noted now so the counter-check and the commit ordering land
together; both are tracked as follow-up alongside open question 5.
Seeking feedback on overall direction, whether
SealedBlobStoreshould live alongside the block-layer abstraction, the
TpmTransportshape (and whether keeping it endpoint-agnostic isthe right call), the libtcgtpm accessor surface, and whether the
Tier-2 runtime re-seal hook belongs in the kernel command path or
should be lifted to the planned user-mode vTPM task.
Commit series
Each commit compiles and passes
cargo clippy --features vtpm(and--features vtpm,vsock,--features vtpm,vtpm-persistwhererelevant) on its own.
make igvmproduces the three IGVM binaries(
coconut-qemu.igvm,coconut-hyperv.igvm,coconut-vanadium.igvm)at HEAD with both
FEATURES=vtpmandFEATURES=vtpm,vtpm-persist.End-to-end validated on an AMD EPYC server with SEV-SNP enabled,
against two TPM 2.0 endpoints sharing the same IGVM binary
(SHA-256
6c1b09cd...):VSOCK→socat→TCP:2321): functional correctness baseline.
(vendor "IFX"/"SLB9672", fw 1.84) on a Gigabyte CTM012 carrier,
reached via
/dev/tpmrm0and a host-side TPM-over-VSOCK bridgedaemon that replaces socat/swtpm on the host side.
Both runs exercise the same L0–L5 guest validation suite and the
same Provision → cold-boot → Recover sequence:
qualified name are byte-identical between Provision and
Recover (
000bdb93ab66.../000b0ae7f061...on SLB 9672;000b92934628.../000bfca90d16...on swtpm).0x1500016written under Provision is readable after acold reboot under Recover (Tier-2 / A3 path).
TPM2_Shutdown(SU_STATE)and commits a fresh SealedBlobbefore SVSM exits (observed in serial log on both endpoints).
Sealed-blob sizes match across backends (~19.1 KB Provision,
~18.8 KB Recover; the observed sub-kilobyte delta between the two
cold-boot samples is consistent with an NV-section repack and is
not endpoint-specific). On SLB 9672 the external-TPM-side latency
is dominated by RSA-2048
TPM2_CreatePrimary(~6.6 s per call onthis device); the critical unseal step itself measures ~126 ms.
Per-command latency CSV (bridge-side wall clock) is included in
the artifact dump.
Reproduction artifacts (IGVM SHA-256, Launch Digest, boot/validate
logs, sealed blobs, TPM-endpoint latency CSV) available on request.
All commits are DCO-signed.