Skip to content

Commit eb9d4d1

Browse files
Managed Elastic Agent through helm chart install updated (#1370)
Main changes: - TOC updated to include the Helm based examples as childs of "Install on Kubernetes using Helm" section (eventually we will organize better the hierarchy of the `Install Elastic Agents in a containerized environment` element, which has too many children. - Removed tech-preview banner in the 3 docs about helm based installation (the chart is currently GA based on @pkoutsovasilis shared details). - Major updates of [Example: Install Fleet-managed Elastic Agent on Kubernetes using Helm](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/1370/reference/fleet/example-kubernetes-fleet-managed-agent-helm) Closes elastic/ingest-docs#1531 --------- Co-authored-by: shainaraskas <[email protected]>
1 parent b8c673f commit eb9d4d1

5 files changed

+123
-62
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
Before installing, add the Elastic Helm repository to your local Helm configuration and verify the available versions of the `elastic-agent` chart. If the repository is already configured, run `helm repo update` to ensure you have the latest package information.
2+
3+
1. Set up Helm repository:
4+
5+
```sh
6+
helm repo add elastic https://helm.elastic.co/
7+
helm repo update
8+
```
9+
10+
2. Verify chart versions:
11+
12+
```sh
13+
helm search repo elastic-agent --versions
14+
```
15+
16+
The previous command returns something similar to:
17+
18+
```sh
19+
NAME CHART VERSION APP VERSION DESCRIPTION
20+
elastic/elastic-agent 9.0.0 9.0.0 Elastic-Agent Helm Chart
21+
elastic/elastic-agent 8.18.0 8.18.0 Elastic-Agent Helm Chart
22+
```
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,118 @@
11
---
22
mapped_pages:
33
- https://www.elastic.co/guide/en/fleet/current/example-kubernetes-fleet-managed-agent-helm.html
4+
applies_to:
5+
stack: ga 8.18
6+
serverless: all
47
---
58

69
# Example: Install Fleet-managed Elastic Agent on Kubernetes using Helm [example-kubernetes-fleet-managed-agent-helm]
710

8-
::::{warning}
9-
This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
10-
::::
11-
11+
This example demonstrates how to install a {{fleet}}-managed {{agent}} on a {{k8s}} system using a Helm chart, collect {{k8s}} metrics and logs using the [Kubernetes Integration](integration-docs://reference/kubernetes/index.md), and send the data to an {{es}} cluster in {{ecloud}} for visualization in {{kib}}.
1212

13-
This example demonstrates how to install {{fleet}}-managed {{agent}} on a {{k8s}} system using a Helm chart, gather {{k8s}} metrics and send them to an {{es}} cluster in {{ecloud}}, and then view visualizations of those metrics in {{kib}}.
13+
Although this tutorial uses an {{ech}} deployment, the same steps can be adapted for other deployment types. For self-managed, {{eck}}, or {{ece}} deployments, you might need to provide the {{fleet}} Server CA certificate during the {{agent}} installation, as outlined below.
1414

1515
For an overview of the {{agent}} Helm chart and its benefits, refer to [Install {{agent}} on Kubernetes using Helm](/reference/fleet/install-on-kubernetes-using-helm.md).
1616

1717
This guide takes you through these steps:
1818

19-
* [Install {{agent}}](#agent-fleet-managed-helm-example-install-agent)
19+
* [Add the Elastic Helm repository](#preparations)
20+
* [Create a {{fleet}} policy and install {{agent}}](#agent-fleet-managed-helm-example-install-agent)
2021
* [Install the Kubernetes integration](#agent-fleet-managed-helm-example-install-integration)
2122
* [Tidy up](#agent-fleet-managed-helm-example-tidy-up)
2223

23-
2424
## Prerequisites [agent-fleet-managed-helm-example-prereqs]
2525

2626
To get started, you need:
2727

2828
* A local install of the [Helm](https://helm.sh/) {{k8s}} package manager.
29-
* An [{{ecloud}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body) hosted {{es}} cluster on version 8.16 or higher.
29+
* An [{{ech}}](https://cloud.elastic.co/registration?page=docs&placement=docs-body) {{es}} cluster on version 8.18 or higher, with an [Integrations Server](/deploy-manage/deploy/elastic-cloud/ec-customize-deployment-components.md#ec_integrations_server) component. An {{serverless-full}} project also meets this requirement.
3030
* An active {{k8s}} cluster.
31-
* A local clone of the [elastic/elastic-agent](https://github.com/elastic/elastic-agent/tree/8.16) GitHub repository. Make sure to use the `8.16` branch to ensure that {{agent}} has full compatibility with the Helm chart.
3231

32+
## Installation overview [overview]
33+
34+
The installation and configuration steps shown in this example deploys the following components to monitor your Kubernetes cluster:
35+
36+
* A default installation of [`kube-state-metrics` (KSM)](https://github.com/kubernetes/kube-state-metrics), configured as a dependency of the Helm chart. KSM is required by the Kubernetes integration to collect cluster-level metrics.
37+
38+
* A group of {{agent}}s deployed as a [Kubernetes DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/), connected to {{fleet}}, and configured through a {{fleet}} policy to collect the following metrics and logs:
39+
* Host-level metrics and logs through the [System integration](integration-docs://reference/system/index.md): This enables the monitoring of your Kubernetes nodes at OS level. {{agent}} Pods will collect system metrics and logs from their own hosts.
40+
* Kubernetes metrics and logs through the [Kubernetes integration](integration-docs://reference/kubernetes/index.md): This enables Kubernetes monitoring at both cluster and node levels. All {{agent}} Pods will collect node-level Kubernetes metrics and logs from their own hosts, while one of the agents will also collect cluster-level metrics and events, acting as a [leader](./kubernetes_leaderelection-provider.md).
41+
42+
By default, all resources are installed in the namespace defined by your current `kubectl` context. You can override this by specifying a different namespace using the `--namespace` option during installation.
43+
44+
::::{note}
45+
The proposed approach of a single {{agent}} DaemonSet to collect all metrics works well for small to medium-sized {{k8s}} clusters.
46+
47+
For larger clusters, or when kube-state-metrics (KSM) metrics collection becomes a performance bottleneck, we recommend a more scalable architecture: move the KSM metric collection to a separate set of agents deployed as sidecars alongside KSM, with autosharding enabled.
48+
49+
This can be easily implemented with the Helm chart. For details, refer to the [KSM autosharding example](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-ksm-sharding).
50+
::::
51+
52+
53+
% we will uncomment the next line when the use cases are documented in the landing page :)
54+
% For other architectures and use cases, refer to [Advanced use cases](./install-on-kubernetes-using-helm.md#advanced-use-cases).
55+
56+
## Step 1: Add the Elastic Helm repository [preparations]
3357

34-
## Install {{agent}} [agent-fleet-managed-helm-example-install-agent]
58+
:::{include} _snippets/agent_add_helm_repository.md
59+
:::
60+
61+
## Step 2: Create a {{fleet}} policy and install {{agent}} [agent-fleet-managed-helm-example-install-agent]
3562

3663
1. Open your {{ecloud}} deployment, and from the navigation menu select **Fleet**.
3764
2. From the **Agents** tab, select **Add agent**.
3865
3. In the **Add agent** UI, specify a policy name and select **Create policy**. Leave the **Collect system logs and metrics** option selected.
3966
4. Scroll down in the **Add agent** flyout to the **Install Elastic Agent on your host** section.
4067
5. Select the **Linux TAR** tab and copy the values for `url` and `enrollment-token`. You’ll use these when you run the `helm install` command.
41-
6. Open a terminal shell and change into a directory in your local clone of the `elastic-agent` repo.
42-
7. Copy this command.
68+
6. Open a terminal shell on your local system where the Helm tool is installed and you have access to the {{k8s}} cluster.
69+
7. Copy and prepare the command to install the chart:
4370

4471
```sh
45-
helm install demo ./deploy/helm/elastic-agent \
72+
helm install demo elastic/elastic-agent \
4673
--set agent.fleet.enabled=true \
47-
--set agent.fleet.url=<Fleet-URL> \
48-
--set agent.fleet.token=<Fleet-token> \
74+
--set system.enabled=true \
75+
--set agent.fleet.url=<Fleet-URL> \ # Substitute Fleet-URL with the URL that you copied earlier
76+
--set agent.fleet.token=<Fleet-token> \ # Substitute Fleet-token with the enrollment token that you copied earlier.
4977
--set agent.fleet.preset=perNode
5078
```
5179

52-
Note that the command has these properties:
53-
54-
* `helm install` runs the Helm CLI install tool.
55-
* `demo` gives a name to the installed chart. You can choose any name.
56-
* `./deploy/helm/elastic-agent` is a local path to the Helm chart to install (in time it’s planned to have a public URL for the chart).
57-
* `--set agent.fleet.enabled=true` enables {{fleet}}-managed {{agent}}. The CLI parameter overrides the default `false` value for `agent.fleet.enabled` in the {{agent}} [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml) file.
58-
* `--set agent.fleet.url=<Fleet-URL>` sets the address where {{agent}} will connect to {{fleet}} in your {{ecloud}} deployment, over port 443 (again, overriding the value set by default in the {{agent}} [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml) file).
59-
* `--set agent.fleet.token=<Fleet-token>` sets the enrollment token that {{agent}} uses to authenticate with {{fleet}}.
60-
* `--set agent.fleet.preset=perNode` enables {{k8s}} metrics on `per node` basis. You can alternatively set cluster wide metrics (`clusterWide`) or kube-state-metrics (`ksmSharded`).
80+
The command has these properties:
6181

62-
::::{tip}
63-
For a full list of all available YAML settings and descriptions, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent).
64-
::::
82+
* `helm install`: Runs the Helm CLI install tool. You can use `helm upgrade` to modify or update an installed release.
83+
* `demo`: The name for this specific installation of the chart, known as the **release name**. You can choose any name you like.
84+
* `elastic/elastic-agent`: The name of the chart to install, using the format `<repository>/<chart-name>`.
85+
* `--set agent.fleet.enabled=true`: Enables {{fleet}}-managed {{agent}}, which is disabled (`false`) by default.
86+
* `--set system.enabled=true`: Adds the required volumes and mounts to enable host monitoring through the System integration.
87+
* `--set agent.fleet.url=<Fleet-URL>`: Specifies the address where {{agent}} connects to {{fleet}} Server in your {{ecloud}} deployment.
88+
* `--set agent.fleet.token=<Fleet-token>`: Sets the enrollment token that {{agent}} uses to authenticate with {{fleet}} Server.
89+
* `--set agent.fleet.preset=perNode`: Runs the agent as a `DaemonSet`, which is required for the purpose of this example. Refer to [](install-on-kubernetes-using-helm.md) for more details and use cases for this parameter.
6590

66-
8. Update the command to replace:
91+
After your updates, the command should be similar to:
6792

68-
1. `<Fleet-URL>` with the URL that you copied earlier.
69-
2. `<Fleet-token>` with the enrollment token that you copied earlier.
93+
```sh
94+
helm install demo elastic/elastic-agent \
95+
--set agent.fleet.enabled=true \
96+
--set system.enabled=true \
97+
--set agent.fleet.url=https://256575858845283fxxxxxxxd5265d2b4.fleet.us-central1.gcp.foundit.no:443 \
98+
--set agent.fleet.token=eSVvFDUvSUNPFldFdhhZNFwvS5xxxxxxxxxxxxFEWB1eFF1YedUQ1NWFXwr== \
99+
--set agent.fleet.preset=perNode
100+
```
70101

71-
After your updates, the command should look something like this:
102+
::::{tip}
103+
For a full list of all available values settings and descriptions, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent) and default [values.yaml](https://github.com/elastic/elastic-agent/blob/main/deploy/helm/elastic-agent/values.yaml).
72104

73-
```sh
74-
helm install demo ./deploy/helm/elastic-agent \
75-
--set agent.fleet.enabled=true \
76-
--set agent.fleet.url=https://256575858845283fxxxxxxxd5265d2b4.fleet.us-central1.gcp.foundit.no:443 \
77-
--set agent.fleet.token=eSVvFDUvSUNPFldFdhhZNFwvS5xxxxxxxxxxxxFEWB1eFF1YedUQ1NWFXwr== \
78-
--set agent.fleet.preset=perNode
79-
```
105+
The following options could be useful for special use cases:
106+
* `--namespace <namespace>`: Allows to install all resources in a specific namespace.
107+
* `--version <version>`: Installs a specific version of the Helm chart and {{agent}}. Refer to [Preparations](#preparations) to check available versions.
108+
* `--set agent.version=<version>`: Installs a specific version of {{agent}}. By default, the chart installs the agent version that matches its own.
109+
* `--set-file agent.fleet.ca.value=/local-path/to/fleet-ca.crt`: Provides the CA certificate used by the {{fleet}} Server. This is typically needed when the server uses a certificate signed by a private CA. Not required for {{fleet}} Servers running on {{ecloud}}.
110+
* `--set agent.fleet.insecure=true`: Use this option to skip the {{fleet}} certificate verification if your {{fleet}} Server uses a self-signed certificate, such as when installed in quickstart mode. Not required for {{fleet}} Servers running on {{ecloud}}. Note that this option is not recommended for production environments.
111+
* `--set kube-state-metrics.enabled=false`: In case you already have KSM installed in your cluster, and you don't want to install a second instance.
112+
* `--set kube-state-metrics.fullnameOverride=ksm`: If you want to deploy KSM with a different release name (it defaults to `kube-state-metrics`). This can be useful if you have already a default installation of KSM and you want a second one.
113+
::::
80114
81-
9. Run the command.
115+
8. Run the command.
82116
83117
The command output should confirm that {{agent}} has been installed:
84118
@@ -89,32 +123,36 @@ To get started, you need:
89123
...
90124
```
91125
92-
10. Run the `kubectl get pods -n default` command to confirm that the {{agent}} pod is running:
126+
9. Run the `kubectl get pods -n default` command to confirm that the {{agent}} Pods are running. You should see one {{agent}} Pod running on each Kubernetes node:
93127
94128
```sh
95129
NAME READY STATUS RESTARTS AGE
96130
agent-pernode-demo-86mst 1/1 Running 0 12s
97131
```
98132
99-
11. In the **Add agent** flyout, wait a minute or so for confirmation that {{agent}} has successfully enrolled with {{fleet}} and that data is flowing:
133+
::::{note}
134+
If your Kubernetes nodes have [`taints`](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) configured, you may need to add `tolerations` to the {{agent}} DaemonSet during installation to ensure the Pods can run on tainted nodes.
135+
136+
You can do this by setting the `presets.perNode.tolerations[]` value, which accepts standard Kubernetes toleration definitions.
137+
::::
138+
139+
10. In the **Add agent** flyout, wait a minute or so for confirmation that {{agent}} has successfully enrolled with {{fleet}} and that data is flowing:
100140
101141
:::{image} images/helm-example-nodes-enrollment-confirmation.png
102142
:alt: Screen capture of Add Agent UI showing that the agent has enrolled in Fleet
103143
:screenshot:
104144
:::
105145
106-
12. In {{fleet}}, open the **Agents** tab and see that an **Agent-pernode-demo-#** agent is running.
107-
13. Select the agent to view its details.
108-
14. On the **Agent details** tab, on the **Integrations** pane, expand `system-1` to confirm that logs and metrics are incoming. You can click either the `Logs` or `Metrics` link to view details.
146+
11. In {{fleet}}, open the **Agents** tab and see that an **Agent-pernode-demo-#** agent is running.
147+
12. Select the agent to view its details.
148+
13. On the **Agent details** tab, on the **Integrations** pane, expand `system-1` to confirm that logs and metrics are incoming. You can click either the `Logs` or `Metrics` link to view details.
109149
110150
:::{image} images/helm-example-nodes-logs-and-metrics.png
111151
:alt: Screen capture of the Logs and Metrics view on the Integrations pane
112152
:screenshot:
113153
:::
114154
115-
116-
117-
## Install the Kubernetes integration [agent-fleet-managed-helm-example-install-integration]
155+
## Step 3: Install the Kubernetes integration [agent-fleet-managed-helm-example-install-integration]
118156
119157
Now that you’ve {{agent}} and data is flowing, you can set up the {{k8s}} integration.
120158
@@ -123,9 +161,15 @@ Now that you’ve {{agent}} and data is flowing, you can set up the {{k8s}} inte
123161
3. On the {{k8s}} integration page, click **Add Kubernetes** to add the integration to your {{agent}} policy.
124162
4. Scroll to the bottom of **Add Kubernetes integration** page. Under **Where to add this integration?** select the **Existing hosts** tab. On the **Agent policies** menu, select the agent policy that you created previously in the [Install {{agent}}](#agent-fleet-managed-helm-example-install-agent) steps.
125163
126-
You can leave all of the other integration settings at their default values.
164+
You can leave all of the other integration settings at their default values. For details about the available inputs and data sets, refer to the [Kubernetes integration](integration-docs://reference/kubernetes/index.md) documentation.
165+
166+
::::{important}
167+
All inputs under the **Collect Kubernetes metrics from kube-state-metrics** section default to `kube-state-metrics:8080` as the destination host. This works if you deployed KSM (kube-state-metrics) alongside {{agent}} during the chart installation, which is the default behavior, as both components are installed in the same namespace.
127168
128-
5. Click **Save and continue**. When prompted, select to **Add Elastic Agent later** since you’ve already added it using Helm.
169+
If your KSM instance runs in a different namespace than {{agent}}, or if it uses a different service name, update the `host` setting in each data set of the integration to point to the KSM service.
170+
::::
171+
172+
5. Click **Save and continue**. When prompted, select **Add Elastic Agent later**, because you’ve already added the agent using Helm.
129173
6. On the {{k8s}} integration page, open the **Assets** tab and select the **[Metrics Kubernetes] Pods** dashboard.
130174
131175
On the dashboard, you can view the status of your {{k8s}} pods, including metrics on memory usage, CPU usage, and network throughput.
@@ -153,4 +197,8 @@ The uninstall should be confirmed as shown:
153197
release "demo" uninstalled
154198
```
155199
156-
As a reminder, for full details about using the {{agent}} Helm chart refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent).
200+
## Next steps
201+
202+
For full details about using the {{agent}} Helm chart, refer to the [{{agent}} Helm Chart Readme](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent).
203+
204+
Refer to the [examples](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples) section of the GitHub repository for advanced use cases, such as integrating {{agent}}s with [KSM autosharding](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-ksm-sharding), or configuring [mutual TLS authentication](https://github.com/elastic/elastic-agent/tree/main/deploy/helm/elastic-agent/examples/fleet-managed-certificates) between {{agent}}s and the {{fleet}} Server.

reference/fleet/example-kubernetes-standalone-agent-helm.md

-5
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,6 @@ mapped_pages:
55

66
# Example: Install standalone Elastic Agent on Kubernetes using Helm [example-kubernetes-standalone-agent-helm]
77

8-
::::{warning}
9-
This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
10-
::::
11-
12-
138
This example demonstrates how to install standalone {{agent}} on a Kubernetes system using a Helm chart, gather Kubernetes metrics and send them to an {{es}} cluster in {{ecloud}}, and then view visualizations of those metrics in {{kib}}.
149

1510
For an overview of the {{agent}} Helm chart and its benefits, refer to [Install {{agent}} on Kubernetes using Helm](/reference/fleet/install-on-kubernetes-using-helm.md).

0 commit comments

Comments
 (0)