Adding SPIFFE/SPIRE support to CoCo/Trustee is tricky, but I think we finally have a pretty good understanding of how to do this.
My basic proposal is to add a plugin to Trustee that can generate SVIDs and provide them to a workload. This will skip over the two-tiered SPIRE attestor design, which isn't a great fit for confidential computing (where the node is untrusted). Rather than generating SVIDs completely from scratch and managing workload state, the plugin will use SPIRE's delegated identity API to connect to an existing SPIRE server. This approach will only provide pod-level selectors. Container-level selectors require additional logic inside the podvm, which is out of scope for now.
Taking a step back, we have had a couple previous proposals for SPIFFE support. First, there has been some work to put the SPIRE Agent itself into the podvm. This has some drawbacks. We end up with a lot of "nodes" and the k8s workload attestor doesn't work inside a podvm without major alterations. We have also discussed integrating SPIFFE directly into the KBS protocol. This is a much more involved effort and I think we are currently leaning more towards mTLS for this. In any case, adding a SPIRE plugin does not preclude future development.
Delegated Identity API
The delegated identity API is an interface of the SPIRE agent. It can take a list of selectors and return the corresponding SVID. This is perfect for us, since we can skip over the entire attestor flow and come up with the selectors our own way (described below), but we can still use the existing SPIRE server. The snag is that to use the delegated identity API, a client needs its own SVID and there needs to be a SPIRE agent running locally. This means that we will need to do some bootstrapping (described below). Although this is a bit inconvenient, it is actually fairly elegant. The real SPIRE agent is going to delegate its powers to the Trustee plugin. As such, Trustee should be attested by SPIRE.
Selectors
The selectors will come from the init-data / validated-identifiers. We have already laid the groundwork for this. Basically the selectors will stem from the measured configurations in the init-data (such as the agent policy and image-signature-policy). We can use the validated-identifiers extension to map these to more standard fields, which the plugin can use as selectors. We should be able to approximate typical k8s pod-level selectors, although not all fields will necessarily be measured. We will first focus on the mechanics of the plugin. The policies and mappings can be refined as we get more requirements.
Bootstrap
This proposal mainly targets users that already have some kind of SPIRE setup and want to add CoCo workloads to it without too many changes. For simplicity, let's assume that we will deploy Trustee with the Helm chart. Here are the steps for the first deployment.
- Trustee Helm chart deployed on a node that already has SPIRE support (SPIRE agent + k8s attestor)
- SPIRE server has a workload entry for Trustee (KBS) created
- SPIRE Plugin in KBS retrieves SVID
Runtime
Here's the flow at runtime
- Workload created with kata gent policy in init-data
- guest workload requests SVID from the local rest API (ASR)
- attestation service creates attestation token for guest (including validated identifiers)
- KBS policy ensures that attestation token is affirming (guest tcb is valid)
- plugin extracts selectors from validated-idenfitifiers / init-data
- plugin uses delegated identity API to get SVID for selectors
- KBS forwards (wrapped) SVID back to pod
When the SVID expires, repeat steps 1-6.
Adding SPIFFE/SPIRE support to CoCo/Trustee is tricky, but I think we finally have a pretty good understanding of how to do this.
My basic proposal is to add a plugin to Trustee that can generate SVIDs and provide them to a workload. This will skip over the two-tiered SPIRE attestor design, which isn't a great fit for confidential computing (where the node is untrusted). Rather than generating SVIDs completely from scratch and managing workload state, the plugin will use SPIRE's delegated identity API to connect to an existing SPIRE server. This approach will only provide pod-level selectors. Container-level selectors require additional logic inside the podvm, which is out of scope for now.
Taking a step back, we have had a couple previous proposals for SPIFFE support. First, there has been some work to put the SPIRE Agent itself into the podvm. This has some drawbacks. We end up with a lot of "nodes" and the k8s workload attestor doesn't work inside a podvm without major alterations. We have also discussed integrating SPIFFE directly into the KBS protocol. This is a much more involved effort and I think we are currently leaning more towards mTLS for this. In any case, adding a SPIRE plugin does not preclude future development.
Delegated Identity API
The delegated identity API is an interface of the SPIRE agent. It can take a list of selectors and return the corresponding SVID. This is perfect for us, since we can skip over the entire attestor flow and come up with the selectors our own way (described below), but we can still use the existing SPIRE server. The snag is that to use the delegated identity API, a client needs its own SVID and there needs to be a SPIRE agent running locally. This means that we will need to do some bootstrapping (described below). Although this is a bit inconvenient, it is actually fairly elegant. The real SPIRE agent is going to delegate its powers to the Trustee plugin. As such, Trustee should be attested by SPIRE.
Selectors
The selectors will come from the init-data / validated-identifiers. We have already laid the groundwork for this. Basically the selectors will stem from the measured configurations in the init-data (such as the agent policy and image-signature-policy). We can use the validated-identifiers extension to map these to more standard fields, which the plugin can use as selectors. We should be able to approximate typical k8s pod-level selectors, although not all fields will necessarily be measured. We will first focus on the mechanics of the plugin. The policies and mappings can be refined as we get more requirements.
Bootstrap
This proposal mainly targets users that already have some kind of SPIRE setup and want to add CoCo workloads to it without too many changes. For simplicity, let's assume that we will deploy Trustee with the Helm chart. Here are the steps for the first deployment.
Runtime
Here's the flow at runtime
When the SVID expires, repeat steps 1-6.