You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,6 +23,10 @@ These pages detail the components and how to configure the EDOT Collector.
23
23
-[Manual configurations](docs/manual-configuration.md): Manually configure the EDOT Collector to send data to Elastic Observability.
24
24
-[Limitations](docs/collector-limitations.md): Understand the current limitations of the EDOT Collector.
25
25
26
+
## Kubernetes Observability using the EDOT Collector
27
+
28
+
-[Kubernetes guided onboarding](docs/onboarding/8_16/operator/README.md): Use the guided onboarding to send Kubernetes logs, metrics, and application traces to Elasticsearch using the EDOT Collector and [OpenTelemetry Operator]((https://github.com/open-telemetry/opentelemetry-operator/).
29
+
26
30
## Collect application data using the EDOT language SDKs
27
31
28
32
Elastic offers several Distributions that extend [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). The following languages are currently available:
# Get started with OpenTelemetry for Kubernetes Observability
2
2
3
-
This guide will help you get up and running with Kubernetes by walking through the setup and integration of key components, starting with the [OpenTelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator/). The OpenTelemetry Operator is an implementation of a [Kubernetes Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/).
3
+
This guide describes how to:
4
4
5
-
The operator manages:
5
+
- Install the [OpenTelemetry Operator](https://github.com/open-telemetry/opentelemetry-operator/) using the [kube-stack Helm Chart](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-kube-stack).
6
+
- Use the EDOT Collector to send Kubernetes logs, metrics, and application traces to an Elasticsearch cluster.
7
+
- Use the operator for applications [auto-instrumentation](https://opentelemetry.io/docs/kubernetes/operator/automatic/) in all supported languages.
-[auto-instrumentation](https://opentelemetry.io/docs/concepts/instrumentation/automatic/) of the workloads using OpenTelemetry instrumentation libraries
The [kube-stack Helm Chart](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-kube-stack) will be utilized to manage the installation of the Operator's Custom Resource Definitions (CRDs) alongside the configuration of a suite of collectors, which will instrument various components of the Kubernetes environment to enable comprehensive observability and monitoring.
20
+
## Prerequisites
13
21
14
-
### Daemonset collectors
22
+
- Elastic Stack (self-managed or [Elastic Cloud](https://www.elastic.co/cloud)) version 8.16.0 or higher, or an [Elasticsearch serverless](https://www.elastic.co/docs/current/serverless/elasticsearch/get-started) project.
23
+
24
+
- A Kubernetes version supported by the OpenTelemetry Operator (refer to the operator's [compatibility matrix](https://github.com/open-telemetry/opentelemetry-operator?#compatibility-matrix) for more details).
25
+
26
+
## Compatibility Matrix
27
+
28
+
The minimum supported version of the Elastic Stack for OpenTelemetry-based monitoring on Kubernetes is `8.16.0`. Different Elastic Stack releases support specific versions of the [kube-stack Helm Chart](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-kube-stack).
29
+
30
+
The following is the current list of supported versions:
31
+
32
+
| Stack Version | Helm Chart Version | Values file |
When [installing the release](#manual-deployment-of-all-components), ensure you use the right `--version` and `-f <values-file>` parameters. Values files are available in the [resources directory](/resources/kubernetes/operator/helm).
38
+
39
+
## Components description
40
+
41
+
### OpenTelemetry Operator
42
+
43
+
The OpenTelemetry Operator is a [Kubernetes Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) implementation designed to manage OpenTelemetry resources in a Kubernetes environment. It defines and oversees the following Custom Resource Definitions (CRDs):
44
+
45
+
-[OpenTelemetry Collectors](https://github.com/open-telemetry/opentelemetry-collector): Agents responsible for receiving, processing and exporting telemetry data such as logs, metrics, and traces.
46
+
-[Instrumentation](https://opentelemetry.io/docs/kubernetes/operator/automatic): Used for the atomatic instrumentation of workloads by leveraging OpenTelemetry instrumentation libraries.
47
+
48
+
All signals including logs, metrics, traces are processed by the collectors and sent directly to Elasticsearch via the ES exporter. A collector's processor pipeline replaces the traditional APM server functionality for handling application traces.
49
+
50
+
### Kube-stack Helm Chart
51
+
52
+
The [kube-stack Helm Chart](https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-kube-stack) is used to manage the installation of the operator (including its CRDs) and to configure a suite of collectors, which instrument various Kubernetes components to enable comprehensive observability and monitoring.
53
+
54
+
The chart is installed with a provided default `values.yaml` file that can be customized when needed.
55
+
56
+
### DaemonSet collectors
15
57
16
58
The OpenTelemetry components deployed within the DaemonSet collectors are responsible for observing specific signals from each node. To ensure complete data collection, these components must be deployed on every node in the cluster. Failing to do so will result in partial and potentially incomplete data.
17
59
60
+
The DaemonSet collectors handle the following data:
61
+
18
62
- Host Metrics: Collects host metrics (hostmetrics receiver) specific to each node.
19
63
- Kubernetes Metrics: Captures metrics related to the Kubernetes infrastructure on each node.
20
64
- Logs: Utilizes a filelog receiver to gather logs from all Pods running on the respective node.
@@ -24,6 +68,8 @@ The OpenTelemetry components deployed within the DaemonSet collectors are respon
24
68
25
69
The OpenTelemetry components deployed within a Deployment collector focus on gathering data at the cluster level rather than at individual nodes. Unlike DaemonSet collectors, which need to be deployed on every node, a Deployment collector operates as a standalone instance.
26
70
71
+
The Deployment collector handles the following data:
72
+
27
73
- Kubernetes Events: Monitors and collects events occurring across the entire Kubernetes cluster.
28
74
- Cluster Metrics: Captures metrics that provide insights into the overall health and performance of the Kubernetes cluster.
29
75
@@ -37,24 +83,36 @@ The Helm Chart is configured to enable zero-code instrumentation using the [Oper
37
83
- Python
38
84
- .NET
39
85
40
-
Auto-instrumentation is enabled by adding the corresponding annotation to the deployment (or namespace to auto-instrument all pods in the namespace)
86
+
## Deploying components using Kibana Onboarding UX
The preferred method for deploying all components is through the Kibana Onboarding UX. Follow these steps:
47
89
48
-
where <LANGUAGE> is one of: `go` , `java`, `nodejs`, `python`, `dotnet`
90
+
1. Navigate in Kibana to **Observability** --> **Add data**
91
+
2. Select **Kubernetes**, then choose **Kubernetes monitoring with EDOT Collector**.
92
+
3. Follow the on-screen instructions to install the OpenTelemetry Operator using the Helm Chart and the provided `values.yaml`.
93
+
94
+
Notes:
95
+
- If the `elastic_endpoint` showed by the UI is not valid for your environment, replace it with the correct Elasticsearch endpoint.
96
+
- The displayed `elastic_api_key` corresponds to an API key that is automatically generated when the onboarding process is initiated.
97
+
98
+
## Manual deployment of all components
49
99
100
+
### Elastic Stack preparations
50
101
51
-
## Configuration
102
+
Before installing the operator follow these actions:
52
103
53
-
Depending on the deployment model (i.e. self-managed, ESS, serverless), different configuration will be needed.
104
+
1. Create an [API Key](https://www.elastic.co/guide/en/kibana/current/api-keys.html), and make note of its value.
105
+
(TBD: details of API key permissions).
54
106
55
-
### Installation
107
+
2. Install the following integrations in Kibana:
108
+
-`System`
109
+
-`Kubernetes`
110
+
-`Kubernetes OpenTelemetry Assets`
56
111
57
-
All signals including logs, metrics, traces/APM go through the collector directly into Elasticsearch using the ES exporter, a collector's processor pipeline will be used to replace the APM server functionality.
112
+
Notes:
113
+
- When using the [Kibana onboarding UX](#deploying-components-using-kibana-onboarding-ux), the previous actions are automatically handled by Kibana.
114
+
115
+
### Operator Installation
58
116
59
117
1. Create the `opentelemetry-operator-system` Kubernetes namespace:
Regardless of the installation method followed, perform the following checks to verify that everything is running properly:
143
+
144
+
1.**Check Pods Status**
145
+
- Ensure the following components are running without errors:
146
+
-**Operator Pod**
147
+
-**DaemonSet Collector Pod**
148
+
-**Deployment Collector Pod**
149
+
150
+
2.**Validate Instrumentation Object**
151
+
- Confirm that the **Instrumentation object** is deployed and configured with a valid **endpoint**.
152
+
153
+
3.**Kibana Dashboard Check**
154
+
- Verify that the **[OTEL][Metrics Kubernetes] Cluster Overview** dashboard in **Kibana** is displaying data correctly.
155
+
156
+
4.**Log Data Availability in Kibana**
157
+
- In **Kibana Discovery**, confirm the availability of data under the `__logs-*__` data view.
158
+
159
+
5.**Metrics Data Availability in Kibana**
160
+
- In **Kibana Discovery**, ensure data is available under the `__metrics-*__` data view.
161
+
162
+
## Instrumenting Applications
163
+
164
+
To enable auto-instrumentation, add the corresponding annotation to the pods of existing deployments (`spec.template.metadata.annotations`), or to the desired namespace (to auto-instrument all pods in the namespace):
where <LANGUAGE> is one of: `go` , `java`, `nodejs`, `python`, `dotnet`
173
+
174
+
For detailed instructions and examples on how to instrument applications in Kubernetes using the OpenTelemetry Operator, refer to this guide (TBD-add link and document).
0 commit comments