[Helm] Allow providing CA certificate of the Fleet Server when running the agent in fleet mode

[eedugon]

When installing the agent in Fleet Mode with something like:

helm install demo ./deploy/helm/elastic-agent \
--set agent.fleet.enabled=true \
--set agent.fleet.url=https://fleet-svc.default.svc \
--set agent.fleet.token=TTg1NHNaTUJoNkpaNzE4R3IzeGg6WXo2MUxSakJTNmVvZUE3d212V0JGUQ== \
--set agent.fleet.preset=perNode \

If the Fleet Server is configured with a certificate signed by a corporate / custom / intermediate CA the Elastic Agent should get the CA certificate configured on FLEET_CA environment variable (if I'm not mistaken, based on https://www.elastic.co/guide/en/fleet/current/agent-environment-variables.html#env-enroll-agent).

The only current workaround is to use --set agent.fleet.insecure=true to bypass the certificate check during enrollment.

Note that providing the Elasticsearch CA for a normal agent (Fleet Managed) is not needed as it will be fetched from the policy at a later stage.

In my opinion this is important to achieve before considering the helm chart GA (cc: @nimarezainia / @pkoutsovasilis ).

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[nimarezainia]

@pkoutsovasilis I assume that those variables are missing from Helm - correct? which would be required before GA of this.

Does it make sense to ensure that all the options available can be configured via Helm? (referring to these)

[pkoutsovasilis]

Hey @nimarezainia 👋, apologies for the delay - this one slipped off my radar.

To address your question, we’re indeed missing the values to pass inside the container for these variables:

FLEET_CA
KIBANA_CA
ELASTICSEARCH_CA
FLEET_INSECURE
FLEET_TOKEN_NAME
FLEET_TOKEN_POLICY_NAME
FLEET_DAEMON_TIMEOUT
ELASTIC_AGENT_CERT
ELASTIC_AGENT_CERT_KEY

Regarding the proxy extra args, it seems these aren’t supported when running a containerized agent (relevant code reference).

One issue is that variables like FLEET_CA, KIBANA_CA, ELASTICSEARCH_CA, ELASTIC_AGENT_CERT, and ELASTIC_AGENT_CERT_KEY are essentially file paths. Since Helm doesn’t allow users to directly input or read file contents, the workaround is to let users specify their content instead. Then, we can create custom ConfigMaps that the Helm chart can mount, setting up the respective container environment variables accordingly.

[nimarezainia]

One issue is that variables like FLEET_CA, KIBANA_CA, ELASTICSEARCH_CA, ELASTIC_AGENT_CERT, and ELASTIC_AGENT_CERT_KEY are essentially file paths. Since Helm doesn’t allow users to directly input or read file contents, the workaround is to let users specify their content instead. Then, we can create custom ConfigMaps that the Helm chart can mount, setting up the respective container environment variables accordingly.

just a correction, these may be file-paths or actual CA/certs. It would be great if we could support the file path option also.

[strawgate]

I believe at least elasticsearch ca is ignored in fleet mode (which the issue creator mentions), worth checking the other variables as well.

[pkoutsovasilis]

@nimarezainia I indeed do remember some certificate-related env vars that could be either a path or the actual certificate value but for these ones from what I have seen in the code and from what I read in the documentation

FLEET_CA (string) The path to a certificate authority. Overrides ELASTICSEARCH_CA when set.By default, Elastic Agent uses the list of trusted certificate authorities (CA) from the operating system where it is running. If the certificate authority that signed your node certificates is not in the host system’s trusted certificate authorities list, use this config to add the path to the .pem file that contains your CA’s certificate.

KIBANA_CA (string) The path to a certificate authority.By default, Elastic Agent uses the list of trusted certificate authorities (CA) from the operating system where it is running. If the certificate authority that signed your node certificates is not in the host system’s trusted certificate authorities list, use this config to add the path to the .pem file that contains your CA’s certificate.

ELASTICSEARCH_CA (string) The path to a certificate authority.By default, Elastic Agent uses the list of trusted certificate authorities (CA) from the operating system where it is running. If the certificate authority that signed your node certificates is not in the host system’s trusted certificate authorities list, use this config to add the path to the .pem file that contains your CA’s certificate.

ELASTIC_AGENT_CERT (string) The path to the mutual TLS client certificate that Elastic Agent will use to connect to Fleet Server.

ELASTIC_AGENT_CERT_KEY (string) The path to the mutual TLS private key that Elastic Agent will use to connect to Fleet Server.

are paths only? please tell me what I am missing.

Moreover, supporting a user specifying a path during the installation and making this path appear inside a pod with the same contents is not possible as Helm doesn't support reading contents of arbitrary paths. Two possible solutions I can thing of are:

1. allow the users specify already existing configmaps that they have created and which will be mounted by the chart inside the agent container
2. allow the users to specify the contents of each certificate through the chart values.yaml and then the chart will create configmaps from them and mount them inside the agent container

[nimarezainia]

I may be mistaken. @AndersonQ would you know? can these fields have a cert or is it always a path?

[AndersonQ]

Hey folks, sorry for the delay, I was on PTO

I was checking the code, the container command passes the values of the environment variables as cli parameters to the enroll cmd, then the enroll cmd validates the values are absolute paths. So, no the variables and cli parameters must be an absolute path

elastic-agent/internal/pkg/agent/cmd/container.go

Lines 504 to 506 in 17814cc

	if cfg.Fleet.CA != "" {
	args = append(args, "--certificate-authorities", cfg.Fleet.CA)
	}

elastic-agent/internal/pkg/agent/cmd/enroll.go

Lines 103 to 166 in 565d139

	func validateEnrollFlags(cmd *cobra.Command) error {
	ca, _ := cmd.Flags().GetString("certificate-authorities")
	if ca != "" && !filepath.IsAbs(ca) {
	return errors.New("--certificate-authorities must be provided as an absolute path", errors.M("path", ca), errors.TypeConfig)
	}
	cert, _ := cmd.Flags().GetString("elastic-agent-cert")
	if cert != "" && !filepath.IsAbs(cert) {
	return errors.New("--elastic-agent-cert must be provided as an absolute path", errors.M("path", cert), errors.TypeConfig)
	}
	key, _ := cmd.Flags().GetString("elastic-agent-cert-key")
	if key != "" && !filepath.IsAbs(key) {
	return errors.New("--elastic-agent-cert-key must be provided as an absolute path", errors.M("path", key), errors.TypeConfig)
	}
	keyPassphrase, _ := cmd.Flags().GetString("elastic-agent-cert-key-passphrase")
	if keyPassphrase != "" {
	if !filepath.IsAbs(keyPassphrase) {
	return errors.New("--elastic-agent-cert-key-passphrase must be provided as an absolute path", errors.M("path", keyPassphrase), errors.TypeConfig)
	}
	if cert == "" || key == "" {
	return errors.New("--elastic-agent-cert and --elastic-agent-cert-key must be provided when using --elastic-agent-cert-key-passphrase", errors.M("path", keyPassphrase), errors.TypeConfig)
	}
	}
	esCa, _ := cmd.Flags().GetString("fleet-server-es-ca")
	if esCa != "" && !filepath.IsAbs(esCa) {
	return errors.New("--fleet-server-es-ca must be provided as an absolute path", errors.M("path", esCa), errors.TypeConfig)
	}
	esCert, _ := cmd.Flags().GetString("fleet-server-es-cert")
	if esCert != "" && !filepath.IsAbs(esCert) {
	return errors.New("--fleet-server-es-cert must be provided as an absolute path", errors.M("path", esCert), errors.TypeConfig)
	}
	esCertKey, _ := cmd.Flags().GetString("fleet-server-es-cert-key")
	if esCertKey != "" && !filepath.IsAbs(esCertKey) {
	return errors.New("--fleet-server-es-cert-key must be provided as an absolute path", errors.M("path", esCertKey), errors.TypeConfig)
	}
	fCert, _ := cmd.Flags().GetString("fleet-server-cert")
	if fCert != "" && !filepath.IsAbs(fCert) {
	return errors.New("--fleet-server-cert must be provided as an absolute path", errors.M("path", fCert), errors.TypeConfig)
	}
	fCertKey, _ := cmd.Flags().GetString("fleet-server-cert-key")
	if fCertKey != "" && !filepath.IsAbs(fCertKey) {
	return errors.New("--fleet-server-cert-key must be provided as an absolute path", errors.M("path", fCertKey), errors.TypeConfig)
	}
	fTokenPath, _ := cmd.Flags().GetString("fleet-server-service-token-path")
	if fTokenPath != "" && !filepath.IsAbs(fTokenPath) {
	return errors.New("--fleet-server-service-token-path must be provided as an absolute path", errors.M("path", fTokenPath), errors.TypeConfig)
	}
	fToken, _ := cmd.Flags().GetString("fleet-server-service-token")
	if fToken != "" && fTokenPath != "" {
	return errors.New("--fleet-server-service-token and --fleet-server-service-token-path are mutually exclusive", errors.TypeConfig)
	}
	fPassphrase, _ := cmd.Flags().GetString("fleet-server-cert-key-passphrase")
	if fPassphrase != "" && !filepath.IsAbs(fPassphrase) {
	return errors.New("--fleet-server-cert-key-passphrase must be provided as an absolute path", errors.M("path", fPassphrase), errors.TypeConfig)
	}
	fClientAuth, _ := cmd.Flags().GetString("fleet-server-client-auth")
	switch fClientAuth {
	case "none", "optional", "required":
	// NOTE we can split this case if we want to do additional checks when optional or required is passed.
	default:
	return errors.New("--fleet-server-client-auth must be one of [none, optional, required]")
	}
	return nil
	}