Add option to retrieve SNP collateral from disk

[eddyashton]

We currently let each node configure where it retrieves SNP collateral from, with a list of endorsement servers. ACI also provides the collateral at a well-known file location. We should add an option to fetch collateral from there, as its likely the easiest and cheapest way to get it.

There is a risk that this collateral is incorrect, out-dated relative to the retrieved attestation. In that case we need some fall-back path to the existing endorsement servers.

This involves flipping the attestation order - we retrieve the attestation first and then the collateral. With the attestation in-hand, we can extract the TCB and determine locally whether the collateral "should" verify us, attempting to fetch fresher collateral if not. This is different from the current server loop, which retries for various connectivity/timeout errors, but expects that the first collateral it receives should be "correct" for the soon-to-be-retrieved attestation.

[eddyashton]

We can also move both of these processes much earlier in the node lifecycle, to where we're reliably single-threaded and have simple code execution, as a step towards standalone testing.

[eddyashton]

Listing plausible sub-tasks, not all independent:

• Add configuration option to specify the location of this file. We could technically add this before it's fully used, to get the (operator-facing) API change in ASAP.
• Move attestation and collateral fetching earlier in the node lifecycle.
• Fetch collateral from disk location, if configuration tells us to.
  • Check that disk-fetched collateral is fresh-enough - if not, fall back to existing servers.
• Do a less-flaky fetch from the servers, with libcurl rather than a hand-spun HTTP stack.
• Test this collateral fetching in isolation, so we can confirm behaviour/repros without spinning up a full CCF service.

[achamayou]

Importantly, attestation fetching needs to happen before collateral resolution, because attestation information is needed to decide if the collateral is up to date.

[eddyashton]

Dumping notes from my initial investigation here, because I'm catching up on SNP attestation.

Endorsement file

• The relevant collateral here is the snp_endorsements (not uvm_endorsements)
• In practice that's a cert chain - a vcekCert followed by a certificateChain, issued by AMD
• On C-ACI, that's available at $UVM_SECURITY_CONTEXT_DIR/host-amd-cert-base64
• We have an ccf::pal::snpACIReportEndorsements struct specifying the schema for this, though it's unused. But that appears to match the fields we manually extract from the response from THIM:

      else if (response_endpoint.response_is_thim_json)
      {
        auto j = nlohmann::json::parse(data);
        auto vcekCert = j.at("vcekCert").get<std::string>();
        auto certificateChain = j.at("certificateChain").get<std::string>();
        endorsements_pem.insert(
          endorsements_pem.end(), vcekCert.begin(), vcekCert.end());
        endorsements_pem.insert(
          endorsements_pem.end(),
          certificateChain.begin(),
          certificateChain.end());
      }

• So assume that's the same structure and that's how we need to eventually concatenate them, but maybe keep an eye on parsing raw DER from files (as we do for AMD servers) for non-ACI deployments
• The freshness check will be comparing the "tcbm" field from this document with the reported_tcb from our ccf::pal::snp::Attestation

Code flow

This whole fetching call-stack starts inside ccf::NodeState::create(), with a call to initiate_quote_generation(). There are multiple layers of call-backs, but the idea is that that populates ccf::NodeState::quote_info, and eventually calls ccf::NodeState::launch_node. I think we can make that whole stack synchronous with libcurl.

[achamayou]

Closed by #6897