Skip to content

Commit cf72d3a

Browse files
theletterfmlunadia
andauthored
Apply suggestions from code review
Co-authored-by: Miguel Luna <[email protected]>
1 parent 840ec9c commit cf72d3a

File tree

3 files changed

+24
-18
lines changed

3 files changed

+24
-18
lines changed

docs/reference/architecture/hosts_vms.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@ products:
1313

1414
# Hosts and VMs environments
1515

16-
On host or virtual machine environments, deploy local, per-host OpenTelemetry Collector instances at the [edge](index.md#understanding-edge-deployment), close to your data sources.
16+
On host or virtual machine environments, deploy local, per-host OpenTelemetry Collector instances at these [edge](index.md#understanding-edge-deployment) host or VMs where your applications are running.
1717

1818
![VM-Edge](../images/arch-vm-edge.png)
1919

20-
These edge collectors have two main purposes:
20+
Collectors deployed on these edge environments have two main purposes:
2121

2222
1. The collection of local logs and infrastructure metrics. Refer to [this sample config file](https://github.com/elastic/elastic-agent/blob/main/internal/pkg/otel/samples/linux/managed_otlp/platformlogs_hostmetrics.yml) for recommended Collector receiver configurations for hostmetrics and logs.
2323
2. Enriching application telemetry from OTel SDKs that run within the instrumented applications on corresponding hosts with resource information.
@@ -36,23 +36,25 @@ Elastic's Observability solution is technically compatible with edge setups that
3636

3737
![VM-Serverless](../images/arch-vm-serverless.png)
3838

39-
Users can send their OTel data from the edge setup in OTel-native format through OTLP without any additional requirements for self-managed preprocessing of data.
39+
Users can send data direct from the Collectors or SDKs deployed on the edge environment via OTLP without any additional requirements for managing an ingestion layer.
4040

4141
### {{ech}}
4242
```{applies_to}
4343
ess:
4444
stack: preview 9.2
4545
```
4646

47-
{{ech}} provides a [Managed OTLP Endpoint](/reference/motlp.md) for ingestion of OpenTelemetry data. Users can send their OTel data from the edge setup in OTel-native format through OTLP without any additional requirements for self-managed preprocessing of data.
47+
{{ech}} provides a [Managed OTLP Endpoint](/reference/motlp.md) for ingestion of OpenTelemetry data. Users can send data direct from the Collectors or SDKs deployed on the edge environment via OTLP without any additional requirements for managing an ingestion layer.
4848

4949
Alternatively, you can run a self-hosted EDOT Collector in gateway mode to ingest OTel data from the edge setup. The EDOT Collector in gateway mode enriches and pre-aggregates the data before ingesting it directly into {{es}}. If required, you can build your custom, EDOT-like Collector [following these instructions](elastic-agent://reference/edot-collector/custom-collector.md).
5050

5151
![VM-ECH](../images/arch-vm-ech.png)
5252

5353
### Self-managed
5454

55-
In a self-managed deployment scenario, you need to host an EDOT Collector in gateway mode that preprocesses and ingests OTel data from the edge setup into the self-managed {{stack}}.
55+
In a self-managed Elastic deployment, we recommend running an EDOT Collector in gateway mode as a unified ingestion layer for OTLP telemetry from OpenTelemetry collectors or SDKs running at the edge. This gateway can receive all signals (logs, metrics and traces), apply processing as needed, and cover the same use cases that previously required components like APM Server or Logstash.
56+
57+
Depending on your scalability and durability needs, this can be a single collector that scales horizontally, or a chain of collectors where each tier handles a specific concern. For high availability and stronger durability guarantees, you can insert Kafka between tiers so that ingestion is buffered and resilient to downstream outages.
5658

5759
![VM-self-managed](../images/arch-vm-self-managed.png)
5860

docs/reference/architecture/index.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,20 +20,22 @@ The following sections outline the recommended architectures for Elastic Distrib
2020

2121
## Understanding edge deployment
2222

23-
In the context of OpenTelemetry architecture, 'edge' refers to collectors deployed close to your data sources: running on individual hosts, virtual machines, or as daemon sets on Kubernetes nodes. These edge collectors perform two primary functions:
23+
In the context of OpenTelemetry architecture, 'edge' refers to the environment where applications are running: individual hosts, virtual machines, or as daemon sets on Kubernetes nodes. Collectors deployed in these environments perform two primary functions:
2424

2525
1. They gather logs, infrastructure metrics, application traces, profiles, and other telemetry from the local environment.
2626
2. They enrich application telemetry from OpenTelemetry SDKs running on the same host or node with resource metadata, such as host information and Kubernetes attributes.
2727

28-
### Collector flexibility at the edge
28+
### Vendor neutral collector deployments on edge environments
2929

3030
You can use any OpenTelemetry Collector distribution at the edge, including:
3131

3232
- The contrib OpenTelemetry Collector.
3333
- Custom-built Collector distributions.
3434
- The {{edot}} Collector.
35+
- Any shipper capable of sending valid OTLP data.
3536

36-
The only requirement is that edge collectors send data using the OpenTelemetry Protocol (OTLP) to either a gateway Collector or directly to {{product.observability}}.
37+
38+
The only requirement is that collectors deployed on edge environments send data using valid OpenTelemetry Protocol (OTLP) to Elastic Cloud (via managed OTLP endpoint) or to the OTLP receiver of a gateway EDOT collector for self-managed deployments.
3739

3840
While any OTLP-compatible collector works at the edge, using EDOT provides a more streamlined experience with:
3941

@@ -45,12 +47,12 @@ While any OTLP-compatible collector works at the edge, using EDOT provides a mor
4547

4648
The need for an EDOT Collector in gateway mode depends on your Elastic deployment type:
4749

48-
**{{serverless-full}} and {{ech}}**: Edge collectors can send OTLP data directly to the [Managed OTLP Endpoint](/reference/motlp.md) without requiring a Gateway Collector. The Managed OTLP Endpoint handles all necessary preprocessing and data transformation.
50+
**{{serverless-full}} and {{ech}}**: Edge collectors can send OTLP data directly to the [Managed OTLP Endpoint](/reference/motlp.md) without requiring a Gateway Collector. The Managed OTLP Endpoint provides a managed durable and resilient ingestion layer.
4951

50-
**Self-managed, ECE, and ECK deployments**: An EDOT Collector in Gateway mode is required between your edge collectors and {{es}}. The gateway Collector performs essential preprocessing, including:
52+
**Self-managed, ECE, and ECK deployments**: An EDOT Collector in Gateway mode is required as part of the stack, this collector will expose the OTLP endpoint that collectors deployed at the edge can send data to. The gateway Collector handles essential preprocessing, including:
5153

5254
- Data enrichment and transformation.
53-
- Metrics aggregation.
55+
- Metrics aggregation for traces and logs which improves APM UI performance with lower latency.
5456
- Format conversion for optimal storage in {{es}}.
5557

5658
:::{note}

docs/reference/architecture/k8s.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,21 +19,23 @@ The recommended OTel architecture for Kubernetes clusters includes a set of Open
1919

2020
## Daemon form
2121

22-
The Collector in Daemon form is deployed on each Kubernetes node as an [edge collector](index.md#understanding-edge-deployment) (close to your data sources) to collect node-local logs and host metrics.
22+
The Collector as a DaemonSet resource kind, deploys an instance on each Kubernetes node as an [edge collector](index.md#understanding-edge-deployment) (close to your data sources) to collect node-local logs and host metrics.
2323

24-
The daemon collector also receives telemetry from applications instrumented with OTel SDKs and running on corresponding nodes.
24+
The DaemonSet collectors also can funnel and enrich telemetry from applications instrumented with OTel SDKs and running on corresponding nodes.
2525

2626
That Collector enriches the application telemetry with resource information such as host and Kubernetes metadata.
2727

2828
All data is then sent through OTLP to the OTel or EDOT Collector in gateway mode.
2929

30-
## Cluster form
30+
## Cluster Collector
3131

32-
The Collector in Cluster form collects Kubernetes cluster-level metrics and sends them to the OTel or EDOT Gateway Collector using OTLP.
32+
The Cluster collector is a Deployment resource kind that collects Kubernetes cluster-level metrics and sends them to the OTel or EDOT Gateway Collector using OTLP.
33+
34+
This instance of the collector helps with collecting cluster level metrics which otherwise would be duplicated by the DaemonSet instances.
3335

3436
## Gateway form
3537

36-
The OTel or EDOT Collector in gateway form gathers the OTel data from all other collectors and ingests it into the Elastic backend.
38+
The OTel or EDOT Collector gateway deployed on the edge Kubernetes environment is not an ingestion layer to Elastic, it centralizes OTel data from all other collectors to help define a unified data pipeline sending OTLP data into the Elastic backend corresponding OTLP endpoint.
3739

3840
For self-managed, ECE, and ECK deployment models the gateway Collector does some additional preprocessing of data.
3941

@@ -69,11 +71,11 @@ Alternatively, you can run a self-hosted EDOT Collector in gateway mode. The gat
6971

7072
### Self-managed
7173

72-
With a self-managed scenario the gateway Collector ingests data directly into the self-managed {{es}} instance.
74+
With a self-managed deployment the gateway Collector ingests data directly into the self-managed {{es}} instance.
7375

7476
![K8s-self-managed](../images/arch-k8s-self-managed.png)
7577

76-
The gateway Collector does some preprocessing and aggregation of OTel data before ingesting it into {{es}}.
78+
The gateway Collector processes APM data and logs to improve latency on solution UIs before is ingested into {{es}}.
7779

7880
While the Daemon and Cluster collectors, as well as the OTel SDKs, can stay fully vendor agnostic or upstream, the gateway Collector needs to be either an EDOT Collector or a [custom, EDOT-like Collector](elastic-agent://reference/edot-collector/custom-collector.md) containing the [required components and preprocessing pipelines](elastic-agent://reference/edot-collector/config/default-config-k8s.md#direct-ingestion-into-elasticsearch).
7981

0 commit comments

Comments
 (0)