Skip to content

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Oct 31, 2025

Proposed commit message

We're injecting default processors while running in standalone beatreceivers. This is not the case in standalone beats.
We should inject only if we're running under agent (Fixed via #46539).

Benchmarks shows improvement and are close to normal mode. There is still some difference and I'll work on that as a follow-up.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool.

How to test this PR locally

You can test the build from this PR with benchbuilder and compare the results without the changes of this PR.

Related issues

Results

  • Benchmark for standalone filebeat - normal mode.
### Benchmark summary table
| Environment ID | Race Id | Suite Name | Benchmark Name | Benchmark Type | Version | Duration (sec) | EPS | Store Bytes | Bytes/Event | Error |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| xyz | 8dd8bf0e-ef1b-4767-8c29-ad986afea6dd | logs-datastream | filestream-json-1024b-balanced | filebeat | `9.3.0` | `1m1.196885s` | `3268.000000` | `56872890` | `284.3644` |  |
| xyz | 43af615c-4b18-49b9-8ca2-4925c44644a3 | logs-datastream | filestream-json-1024b-throughput | filebeat | `9.3.0` | `45.795869s` | `4367.000000` | `56104542` | `280.5227` |  |
| xyz | d76fedb6-5352-4d62-969e-a7bc1aa14d3e | logs-datastream | filestream-json-1024b-scale | filebeat | `9.3.0` | `1m5.14271s` | `3070.000000` | `56834671` | `284.1734` |  |
| xyz | ae0a33c9-5032-47e3-a1e2-ddf0a8f3696d | logs-datastream | filestream-json-1024b-latency | filebeat | `9.3.0` | `19m12.550807s` | `174.000000` | `58669647` | `293.3482` |  |

_[Open in Kibana](https://ingest-metricstore.kb.us-central1.gcp.cloud.es.io:9243/app/dashboards#/view/ce699545-7f3f-42ce-b0f9-16c51e8b01ef?_g=%28time%3A%28from%3A%272025-10-25T14%3A27%3A07.052Z%27%2Cto%3A%272025-10-25T14%3A50%3A30.908Z%27%29%29&_a=%28filters%3A%21%28%28%27%24state%27%3A%28store%3AappState%29%2Cmeta%3A%28alias%3A%21n%2Cdisabled%3A%21f%2Cfield%3Aenvironment_id%2Cindex%3A%27e80ebe49-6c56-452d-a932-2e9026f9ff65%27%2Ckey%3Aenvironment_id%2Cnegate%3A%21f%2Cparams%3A%28query%3A%27xyz%27%29%2Ctype%3Aphrase%29%2Cquery%3A%28match_phrase%3A%28environment_id%3A%27xyz%27%29%29%29%29%2Cquery%3A%28language%3Akuery%2Cquery%3A%27%27%29%29)_
  • Benchmark for filebeat otel mode (before my changes)
### Benchmark summary table
| Environment ID | Race Id | Suite Name | Benchmark Name | Benchmark Type | Version | Duration (sec) | EPS | Store Bytes | Bytes/Event | Error |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| xyz | 755ead19-ab37-4de9-a54c-9395ec17c43a | logs-datastream | filestream-json-1024b-balanced | filebeat | `9.3.0` | `1m56.138989s` | `1722.000000` | `105533417` | `527.6671` |  |
| xyz | 7a7e1caf-7a92-4396-b0c8-6050d6d40ca0 | logs-datastream | filestream-json-1024b-throughput | filebeat | `9.3.0` | `37.276624s` | `5365.000000` | `107132772` | `535.6639` |  |
| xyz | 467e93ba-5d28-4c1c-99f8-d7e83b84f100 | logs-datastream | filestream-json-1024b-scale | filebeat | `9.3.0` | `1m57.625477s` | `1700.000000` | `104303606` | `521.5180` |  |
| xyz | 16d50a55-4e64-4a30-b84c-64561fa7fb1e | logs-datastream | filestream-json-1024b-latency | filebeat | `9.3.0` | `20m8.961231s` | `165.000000` | `108167271` | `540.8364` |  |

_[Open in Kibana](https://ingest-metricstore.kb.us-central1.gcp.cloud.es.io:9243/app/dashboards#/view/ce699545-7f3f-42ce-b0f9-16c51e8b01ef?_g=%28time%3A%28from%3A%272025-10-25T20%3A12%3A12.283Z%27%2Cto%3A%272025-10-25T20%3A38%3A14.524Z%27%29%29&_a=%28filters%3A%21%28%28%27%24state%27%3A%28store%3AappState%29%2Cmeta%3A%28alias%3A%21n%2Cdisabled%3A%21f%2Cfield%3Aenvironment_id%2Cindex%3A%27e80ebe49-6c56-452d-a932-2e9026f9ff65%27%2Ckey%3Aenvironment_id%2Cnegate%3A%21f%2Cparams%3A%28query%3A%27xyz%27%29%2Ctype%3Aphrase%29%2Cquery%3A%28match_phrase%3A%28environment_id%3A%27xyz%27%29%29%29%29%2Cquery%3A%28language%3Akuery%2Cquery%3A%27%27%29%29)_
  • Benchmark for filebeat otel mode (after my changes)
### Benchmark summary table
| Environment ID | Race Id | Suite Name | Benchmark Name | Benchmark Type | Version | Duration (sec) | EPS | Store Bytes | Bytes/Event | Error |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| xyz | 4eefb649-3ae4-498a-b46d-ad6b77f411b0 | logs-datastream | filestream-json-1024b-balanced | filebeat | `9.3.0` | `1m12.23267s` | `2769.000000` | `66011174` | `330.0559` |  |
| xyz | bfd54408-1075-405b-83f5-d20f06181a0e | logs-datastream | filestream-json-1024b-throughput | filebeat | `9.3.0` | `21.711077s` | `9212.000000` | `68151536` | `340.7577` |  |
| xyz | d9bab3f9-0628-480a-aee2-8e6107219e45 | logs-datastream | filestream-json-1024b-scale | filebeat | `9.3.0` | `1m12.787666s` | `2748.000000` | `65854534` | `329.2727` |  |
| xyz | 97cf4796-c3c6-400b-a5f5-e34d4a1f639e | logs-datastream | filestream-json-1024b-latency | filebeat | `9.3.0` | `19m17.321353s` | `173.000000` | `66767287` | `333.8364` |  |

_[Open in Kibana](https://ingest-metricstore.kb.us-central1.gcp.cloud.es.io:9243/app/dashboards#/view/ce699545-7f3f-42ce-b0f9-16c51e8b01ef?_g=%28time%3A%28from%3A%272025-10-25T19%3A46%3A19.037Z%27%2Cto%3A%272025-10-25T20%3A09%3A43.914Z%27%29%29&_a=%28filters%3A%21%28%28%27%24state%27%3A%28store%3AappState%29%2Cmeta%3A%28alias%3A%21n%2Cdisabled%3A%21f%2Cfield%3Aenvironment_id%2Cindex%3A%27e80ebe49-6c56-452d-a932-2e9026f9ff65%27%2Ckey%3Aenvironment_id%2Cnegate%3A%21f%2Cparams%3A%28query%3A%27xyz%27%29%2Ctype%3Aphrase%29%2Cquery%3A%28match_phrase%3A%28environment_id%3A%27xyz%27%29%29%29%29%2Cquery%3A%28language%3Akuery%2Cquery%3A%27%27%29%29)_
```<hr>This is an automatic backport of pull request #47299 done by [Mergify](https://mergify.com).

…7299)

* chore: only add default processors if running unde ragent

* fix test

* fix test

* tab

* fix race condition

* iterate over keys instead of map

* fix test case

(cherry picked from commit 78f115e)

# Conflicts:
#	x-pack/metricbeat/tests/integration/otel_test.go
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Oct 31, 2025
@mergify mergify bot requested a review from a team as a code owner October 31, 2025 05:41
@mergify
Copy link
Contributor Author

mergify bot commented Oct 31, 2025

Cherry-pick of 78f115e has failed:

On branch mergify/bp/8.19/pr-47299
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 78f115e4f.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   libbeat/beat/info.go
	modified:   libbeat/publisher/processing/default.go
	modified:   x-pack/filebeat/fbreceiver/factory.go
	modified:   x-pack/filebeat/fbreceiver/receiver_test.go
	modified:   x-pack/filebeat/tests/integration/otel_test.go
	modified:   x-pack/libbeat/cmd/instance/beat.go
	modified:   x-pack/libbeat/cmd/instance/beat_test.go
	modified:   x-pack/libbeat/outputs/otelconsumer/otelconsumer.go
	modified:   x-pack/libbeat/outputs/otelconsumer/otelconsumer_test.go
	modified:   x-pack/metricbeat/mbreceiver/factory.go
	modified:   x-pack/metricbeat/mbreceiver/receiver_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   x-pack/metricbeat/tests/integration/otel_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@mergify mergify bot removed the request for review from a team October 31, 2025 05:41
@mergify mergify bot requested review from faec and leehinman October 31, 2025 05:41
@mergify mergify bot added backport conflicts There is a conflict in the backported pull request labels Oct 31, 2025
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 31, 2025
@github-actions github-actions bot added Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team skip-changelog labels Oct 31, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 31, 2025
@mergify
Copy link
Contributor Author

mergify bot commented Nov 3, 2025

This pull request has not been merged yet. Could you please review and merge it @VihasMakwana? 🙏

@VihasMakwana VihasMakwana removed the conflicts There is a conflict in the backported pull request label Nov 4, 2025
@VihasMakwana VihasMakwana enabled auto-merge (squash) November 4, 2025 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants