You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/hybrid-agent-beats-receivers.md
+43Lines changed: 43 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -193,6 +193,7 @@ receivers:
193
193
paths:
194
194
- /var/log/*.log
195
195
type: filestream
196
+
queue.mem.flush.timeout: 0s
196
197
metricbeatreceiver:
197
198
metricbeat:
198
199
modules:
@@ -202,6 +203,7 @@ receivers:
202
203
metricsets:
203
204
- cpu
204
205
module: system
206
+
queue.mem.flush.timeout: 0s
205
207
exporters:
206
208
elasticsearch/_agent-component/default:
207
209
api_key: placeholder
@@ -214,6 +216,24 @@ exporters:
214
216
enabled: true
215
217
mapping:
216
218
mode: bodymap
219
+
220
+
retry:
221
+
enabled: true
222
+
initial_interval: 1s
223
+
max_interval: 1m0s
224
+
max_retries: 3
225
+
226
+
sending_queue:
227
+
enabled: true
228
+
wait_for_result: true
229
+
block_on_overflow: true
230
+
num_consumers: 1
231
+
queue_size: 3200
232
+
batch:
233
+
max_size: 1600
234
+
min_size: 0
235
+
flush_timeout: 10s
236
+
sizer: items
217
237
service:
218
238
pipelines:
219
239
logs:
@@ -223,3 +243,26 @@ service:
223
243
- filebeatreceiver
224
244
- metricbeatreceiver
225
245
```
246
+
247
+
### Beats receivers delivery guarantees in OTel mode
248
+
249
+
When Beat receivers are used in OTel mode, event delivery guarantees depend on the configuration of the OpenTelemetry Collector `sending_queue` and retry settings.
250
+
Unlike standalone Beats, the EDOT pipeline allows users to customize queue behavior through the Collector configuration.
251
+
This flexibility is useful, but it also means that not every option combination is compatible with reliable delivery.
252
+
253
+
Elastic Agent in OTel mode provides an **at least once** delivery guarantee for Beat receivers **only when using the supported `sending_queue` settings described below**.
254
+
These settings mirror Beats pipeline behavior closely enough to preserve durability expectations.
255
+
256
+
If users provide arbitrary `sending_queue` or Beat queue overrides, delivery semantics become **undefined** and **at least once delivery cannot be guaranteed**.
257
+
These combinations are not tested and may result in event loss during backpressure or shutdown.
258
+
259
+
To achieve the intended delivery guarantee, the exporter that receives events from Beat receivers must define a `sending_queue` with the following characteristics:
260
+
261
+
- `enabled: true`: The queue must be active.
262
+
- `wait_for_result: true`: The pipeline must wait for the exporter response before removing events.
263
+
- `block_on_overflow: true`: Prevents event drops when the queue is full.
264
+
- The `batch` configuration must include explicit `max_size`, `min_size`, and `flush_timeout` values to ensure events are grouped and flushed in predictable, controlled batches.
265
+
266
+
Additionally, the retry settings must be enabled on the exporter, using a backoff policy that retries until the operation succeeds. By default, `max_retries` is set to 3, which is how most Beats behave. Standalone Filebeat, however, retries indefinitely. Beats receivers don't support unlimited retries yet, and this is being tracked at https://github.com/elastic/beats/issues/47892.
267
+
268
+
Beat receivers also require the Beat-internal memory queue to run in synchronous mode for delivery guarantees. This is enabled by setting `queue.mem.flush.timeout: 0s` in each receiver configuration, as shown in the example above.
0 commit comments