[beatreceiver] Add e2e test for azure blob storage

[khushijain21]

Proposed commit message

This PR adds e2e test that compares document ingested by azure blob storage input in standalone and otel mode.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None

How to test this PR locally

cd x-pack/filebeat
go test -run TestAzureBlobStorageInputOTelE2E -tags integration ./tests/integration -v 

Related issues

• Partialfix for https://github.com/elastic/ingest-dev/issues/4889

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @khushijain21? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[botelastic]

This pull request doesn't have a Team:<team> label.

[infra-vault-gh-plugin-prod]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[infra-vault-gh-plugin-prod]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[github-actions]

TL;DR

x-pack/filebeat: Go Integration Tests is failing before TestCELInputOTelE2E can start Filebeat/OTel because the PR replaced the CEL-specific exporter template with otelElasticsearchExporterYAML, which renders {{ .Index }} but the CEL test still passes an options struct without an Index field.

Remediation

• In x-pack/filebeat/tests/integration/otel_filebeat_input_test.go, update the CEL test options to include Index string, set it to otelIndex before rendering celOTelConfig, set it to fbIndex before rendering celFilebeatConfig, and change the Filebeat config from index: logs-integration-{{ .Namespace }} to index: {{ .Index }}.
• Validate with cd x-pack/filebeat && go test -run TestCELInputOTelE2E -tags integration ./tests/integration -v.

Investigation details

Root Cause

The new shared helper at x-pack/filebeat/tests/integration/otel_helpers_test.go:25-36 defines otelElasticsearchExporterYAML with logs_index: {{ .Index }}.

TestCELInputOTelE2E now uses that helper at x-pack/filebeat/tests/integration/otel_filebeat_input_test.go:86-105, but its local options type still only defines Namespace, ESURL, Username, Password, ResourceURL, and Program at x-pack/filebeat/tests/integration/otel_filebeat_input_test.go:53-60. The template is executed with that value at x-pack/filebeat/tests/integration/otel_filebeat_input_test.go:108-115, so {{ .Index }} cannot be resolved.

Evidence

• Build: https://buildkite.com/elastic/beats/builds/49122
• Job/step: :ubuntu: x-pack/filebeat: Go Integration Tests
• Key log excerpt:

=== FAIL: x-pack/filebeat/tests/integration TestCELInputOTelE2E (0.00s)
otel_filebeat_input_test.go:108:
Error: Received unexpected error:
template: config:12:23: executing "config" at <.Index>: can't evaluate field Index in type integration.options

The FIPS job artifact excerpt only includes the final suite summary (DONE 2882 tests, 11 skipped, 2 failures) and exit status, not the individual assertion blocks, so the actionable logged root cause is the CEL template mismatch above.

Verification

Not run locally: this integration test requires the x-pack/filebeat integration environment and Elasticsearch, while the current workspace is the base branch rather than PR head.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request