Skip to content

Managed Elastic Agent through helm chart install updated #1370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
May 9, 2025
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 91 additions & 41 deletions reference/fleet/example-kubernetes-fleet-managed-agent-helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,71 @@ mapped_pages:

# Example: Install Fleet-managed Elastic Agent on Kubernetes using Helm [example-kubernetes-fleet-managed-agent-helm]

::::{warning}
This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
::::
This example demonstrates how to install a {{fleet}}-managed {{agent}} on a {{k8s}} system using a Helm chart, collect {{k8s}} metrics and logs using the [Kubernetes Integration](integration-docs://reference/kubernetes/index.md), and send the data to an {{es}} cluster in {{ecloud}} for visualization in {{kib}}.

For an overview of the {{agent}} Helm chart and its benefits, refer to [Install {{agent}} on Kubernetes using Helm](/reference/fleet/install-on-kubernetes-using-helm.md).

This example demonstrates how to install {{fleet}}-managed {{agent}} on a {{k8s}} system using a Helm chart, gather {{k8s}} metrics and send them to an {{es}} cluster in {{ecloud}}, and then view visualizations of those metrics in {{kib}}.
::::{note}
This guide deploys a single set of {{agent}} DaemonSet pods to collect all metrics, which works well for small to medium-sized {{k8s}} clusters.

For an overview of the {{agent}} Helm chart and its benefits, refer to [Install {{agent}} on Kubernetes using Helm](/reference/fleet/install-on-kubernetes-using-helm.md).
For larger clusters, or when kube-state-metrics (KSM) metrics collection becomes a performance bottleneck, we recommend a more scalable architecture: move the KSM metric collection to a separate set of agents deployed as sidecars alongside KSM, with autosharding enabled.

This can be easily implemented with the Helm chart. For details, refer to the [KSM autosharding example](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-ksm-sharding).
::::

This guide takes you through these steps:

* [Set up Helm repository](#preparations)
* [Install {{agent}}](#agent-fleet-managed-helm-example-install-agent)
* [Install the Kubernetes integration](#agent-fleet-managed-helm-example-install-integration)
* [Tidy up](#agent-fleet-managed-helm-example-tidy-up)


## Prerequisites [agent-fleet-managed-helm-example-prereqs]

To get started, you need:

* A local install of the [Helm](https://helm.sh/) {{k8s}} package manager.
* An [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body) hosted {{es}} cluster on version 8.16 or higher.
* An [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body) hosted {{es}} cluster on version 8.18 or higher, with an [Integrations Server](/deploy-manage/deploy/elastic-cloud/ec-customize-deployment-components.md#ec_integrations_server) component included.
* An active {{k8s}} cluster.
* A local clone of the [elastic/elastic-agent](https://github.com/elastic/elastic-agent/tree/8.16) GitHub repository. Make sure to use the `8.16` branch to ensure that {{agent}} has full compatibility with the Helm chart.

## Installation overview

The default installation shown in this example deploys the following components to monitor your Kubernetes cluster:

* {{agent}}, deployed as a `DaemonSet`, connected to {{fleet}}, and configured to collect:
* Host level metrics and logs through the [System Integration](integration-docs://reference/system/index.md).
* Kubernetes node and cluster-level metrics and logs through the [Kubernetes Integration](integration-docs://reference/kubernetes/index.md). For cluster-level metrics collection, one of the agents will act as a [leader](./kubernetes_leaderelection-provider.md).
* A default installation of `kube-state-metrics` (KSM), configured as a dependency of the chart. KSM is required by the Kubernetes integration to collect cluster-level metrics.

By default, all resources are installed in the namespace defined by your current `kubectl` context. You can override this by specifying a different namespace using the `--namespace` option during installation.

% we will uncomment the next line when the use cases are documented in the landing page :)
% For other architectures and use cases, refer to [Advanced use cases](./install-on-kubernetes-using-helm.md#advanced-use-cases).

## Preparations [preparations]

Before installing, add the Elastic Helm repository and verify the available versions of the `elastic-agent` chart. If the repository is already configured, run `helm repo update` to ensure you have the latest package information.

1. Set up Helm repository:

```sh
helm repo add elastic https://helm.elastic.co/
helm repo update
```

2. Verify chart versions:

```sh
helm search repo elastic-agent --versions
```

The previous command returns something similar to:

```sh
NAME CHART VERSION APP VERSION DESCRIPTION
elastic/elastic-agent 9.0.0 9.0.0 Elastic-Agent Helm Chart
elastic/elastic-agent 8.18.0 8.18.0 Elastic-Agent Helm Chart
```

## Install {{agent}} [agent-fleet-managed-helm-example-install-agent]

Expand All @@ -38,45 +78,49 @@ To get started, you need:
3. In the **Add agent** UI, specify a policy name and select **Create policy**. Leave the **Collect system logs and metrics** option selected.
4. Scroll down in the **Add agent** flyout to the **Install Elastic Agent on your host** section.
5. Select the **Linux TAR** tab and copy the values for `url` and `enrollment-token`. You’ll use these when you run the `helm install` command.
6. Open a terminal shell and change into a directory in your local clone of the `elastic-agent` repo.
7. Copy this command.
6. Open a terminal shell on your local system where the Helm tool is installed and you have access to the {{k8s}} cluster.
7. Copy and prepare the command to install the chart:

```sh
helm install demo ./deploy/helm/elastic-agent \
helm install demo elastic/elastic-agent \
--set agent.fleet.enabled=true \
--set agent.fleet.url=<Fleet-URL> \
--set agent.fleet.token=<Fleet-token> \
--set agent.fleet.url=<Fleet-URL> \ # Substitute Fleet-URL with the URL that you copied earlier
--set agent.fleet.token=<Fleet-token> \ # Substitute Fleet-token with the enrollment token that you copied earlier.
--set agent.fleet.preset=perNode
```

Note that the command has these properties:

* `helm install` runs the Helm CLI install tool.
* `demo` gives a name to the installed chart. You can choose any name.
* `./deploy/helm/elastic-agent` is a local path to the Helm chart to install (in time it’s planned to have a public URL for the chart).
* `--set agent.fleet.enabled=true` enables {{fleet}}-managed {{agent}}. The CLI parameter overrides the default `false` value for `agent.fleet.enabled` in the {{agent}} [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml) file.
* `--set agent.fleet.url=<Fleet-URL>` sets the address where {{agent}} will connect to {{fleet}} in your {{ecloud}} deployment, over port 443 (again, overriding the value set by default in the {{agent}} [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml) file).
* `--set agent.fleet.token=<Fleet-token>` sets the enrollment token that {{agent}} uses to authenticate with {{fleet}}.
* `--set agent.fleet.preset=perNode` enables {{k8s}} metrics on `per node` basis. You can alternatively set cluster wide metrics (`clusterWide`) or kube-state-metrics (`ksmSharded`).

::::{tip}
For a full list of all available YAML settings and descriptions, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent).
::::

8. Update the command to replace:

1. `<Fleet-URL>` with the URL that you copied earlier.
2. `<Fleet-token>` with the enrollment token that you copied earlier.

After your updates, the command should look something like this:
* `helm install`: Runs the Helm CLI install tool. Use `helm upgrade` to modify or update an existing release.
* `demo`: The name for this specific installation of the chart, known as the **release name**. You can choose any name you like.
* `elastic/elastic-agent`: The name of the chart to install, using the format `<repository>/<chart-name>`.
* `--set agent.fleet.enabled=true`: Enables {{fleet}}-managed {{agent}}, which is disabled (`false`) by default.
* `--set agent.fleet.url=<Fleet-URL>`: Specifies the address where {{agent}} connects to {{fleet}} Server in your {{ecloud}} deployment.
* `--set agent.fleet.token=<Fleet-token>`: Sets the enrollment token that {{agent}} uses to authenticate with {{fleet}} Server.
* `--set agent.fleet.preset=perNode`: Runs the agent as a `DaemonSet`, to collect node-level metrics and logs. Refer to [](install-on-kubernetes-using-helm.md) for more details and use cases for this parameter.

::::{tip}
For a full list of all available values settings and descriptions, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent) and default [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml).

The following options could be useful for special use cases:
* `--namespace <namespace>`: Allows to install all resources in a specific namespace.
* `--version <version>`: Installs a specific version of the Helm chart and {{agent}}. Refer to [Preparations](#preparations) to check available versions.
* `--set agent.version=<version>`: Installs a specific version of {{agent}}. By default, the chart installs the agent version that matches its own.
* `--set-file agent.fleet.ca.value=/local-path/to/fleet-ca.crt`: Provides the CA certificate used by the {{fleet}} Server. This is typically needed when the server uses a certificate signed by a private CA. Not required for {{fleet}} Servers running on {{ecloud}}.
* `--set agent.fleet.insecure=true`: Use this option to skip the {{fleet}} certificate verification if your {{fleet}} Server uses a self-signed certificate, such as when installed in quickstart mode. Not required for {{fleet}} Servers running on {{ecloud}}. Note that this option is not recommended for production environments.
* `--set kube-state-metrics.enabled=false`: In case you already have KSM installed in your cluster, and you don't want to install a second instance.
* `--set kube-state-metrics.fullnameOverride=ksm`: If you want to deploy KSM with a different release name (it defaults to `kube-state-metrics`). This can be useful if you have already a default installation of KSM and you want a second one.
::::

8. After your updates, the command should be similar to:

```sh
helm install demo ./deploy/helm/elastic-agent \
--set agent.fleet.enabled=true \
--set agent.fleet.url=https://256575858845283fxxxxxxxd5265d2b4.fleet.us-central1.gcp.foundit.no:443 \
--set agent.fleet.token=eSVvFDUvSUNPFldFdhhZNFwvS5xxxxxxxxxxxxFEWB1eFF1YedUQ1NWFXwr== \
--set agent.fleet.preset=perNode
```
```sh
helm install demo elastic/elastic-agent \
--set agent.fleet.enabled=true \
--set agent.fleet.url=https://256575858845283fxxxxxxxd5265d2b4.fleet.us-central1.gcp.foundit.no:443 \
--set agent.fleet.token=eSVvFDUvSUNPFldFdhhZNFwvS5xxxxxxxxxxxxFEWB1eFF1YedUQ1NWFXwr== \
--set agent.fleet.preset=perNode
```

9. Run the command.

Expand Down Expand Up @@ -112,8 +156,6 @@ To get started, you need:
:screenshot:
:::



## Install the Kubernetes integration [agent-fleet-managed-helm-example-install-integration]

Now that you’ve {{agent}} and data is flowing, you can set up the {{k8s}} integration.
Expand All @@ -123,7 +165,13 @@ Now that you’ve {{agent}} and data is flowing, you can set up the {{k8s}} inte
3. On the {{k8s}} integration page, click **Add Kubernetes** to add the integration to your {{agent}} policy.
4. Scroll to the bottom of **Add Kubernetes integration** page. Under **Where to add this integration?** select the **Existing hosts** tab. On the **Agent policies** menu, select the agent policy that you created previously in the [Install {{agent}}](#agent-fleet-managed-helm-example-install-agent) steps.

You can leave all of the other integration settings at their default values.
You can leave all of the other integration settings at their default values. For details about the available inputs and data sets, refer to the [Kubernetes Integration](integration-docs://reference/kubernetes/index.md) documentation.

::::{important}
All inputs under the **Collect Kubernetes metrics from kube-state-metrics** section default to `kube-state-metrics:8080` as the destination host. This works if you deployed KSM (kube-state-metrics) alongside {{agent}} during the chart installation, which is the default behavior, as both components are installed in the same namespace.

If your KSM instance runs in a different namespace than {{agent}}, or if it uses a different service name, update the `host` setting in each data set of the integration to point to the KSM service.
::::

5. Click **Save and continue**. When prompted, select to **Add Elastic Agent later** since you’ve already added it using Helm.
6. On the {{k8s}} integration page, open the **Assets** tab and select the **[Metrics Kubernetes] Pods** dashboard.
Expand Down Expand Up @@ -154,3 +202,5 @@ release "demo" uninstalled
```

As a reminder, for full details about using the {{agent}} Helm chart refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent).

Refer also to the [examples](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples) section of the GitHub repository for advanced use cases, such as integrating {{agent}}s with [KSM autosharding](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-ksm-sharding), or configuring [mutual TLS authentication](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-certificates) between {{agent}}s and the {{fleet}} Server.
5 changes: 0 additions & 5 deletions reference/fleet/example-kubernetes-standalone-agent-helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,6 @@ mapped_pages:

# Example: Install standalone Elastic Agent on Kubernetes using Helm [example-kubernetes-standalone-agent-helm]

::::{warning}
This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
::::


This example demonstrates how to install standalone {{agent}} on a Kubernetes system using a Helm chart, gather Kubernetes metrics and send them to an {{es}} cluster in {{ecloud}}, and then view visualizations of those metrics in {{kib}}.

For an overview of the {{agent}} Helm chart and its benefits, refer to [Install {{agent}} on Kubernetes using Helm](/reference/fleet/install-on-kubernetes-using-helm.md).
Expand Down
5 changes: 0 additions & 5 deletions reference/fleet/install-on-kubernetes-using-helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,6 @@ mapped_pages:

# Install Elastic Agent on Kubernetes using Helm [install-on-kubernetes-using-helm]

::::{warning}
This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
::::


Starting with {{stack}} version 8.16, a Helm chart is available for installing {{agent}} in a Kubernetes environment. A Helm-based install offers several advantages, including simplified deployment, availability in marketplaces, streamlined ugrades, as well as quick rollbacks whenever they’re needed.

Features of the Helm-based {{agent}} install include:
Expand Down
5 changes: 3 additions & 2 deletions reference/fleet/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,9 @@ toc:
- file: elastic-agent-container.md
- file: running-on-kubernetes-managed-by-fleet.md
- file: install-on-kubernetes-using-helm.md
- file: example-kubernetes-standalone-agent-helm.md
- file: example-kubernetes-fleet-managed-agent-helm.md
children:
- file: example-kubernetes-standalone-agent-helm.md
- file: example-kubernetes-fleet-managed-agent-helm.md
- file: advanced-kubernetes-managed-by-fleet.md
- file: configuring-kubernetes-metadata.md
- file: running-on-gke-managed-by-fleet.md
Expand Down
Loading