Skip to content

feat: add support for mapping s3 bucket prefix to OTel resource attributes #39634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

yshengli
Copy link

@yshengli yshengli commented Apr 24, 2025

Description

Introduced the resource_attrs_to_s3/s3_prefix configuration field to enable mapping the S3 bucket prefix to an OpenTelemetry resource attribute. When this field is set, the S3 prefix is determined by the specified resource attribute, with s3uploader/s3_prefix used as a fallback if the attribute is missing or empty.

Link to tracking issue

Fixes #37858

Testing

  • Added unit tests to verify the functionality of the related code changes
  • Built the OpenTelemetry Collector locally to ensure the component behaves as intended

Documentation

Updated the README to document the new configuration field:

  • Described its impact on the component’s behavior
  • Clarified how it interacts with the s3uploader/s3_prefix setting
  • Included an example demonstrating how to use it

@yshengli yshengli requested review from atoulme and a team as code owners April 24, 2025 18:03
Copy link

linux-foundation-easycla bot commented Apr 24, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

Copy link
Contributor

@Erog38 Erog38 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One change I ask is around choosing which attribute to map, and using both resource as well as metric, log, and span attributes.

Other than that, nice work!

@atoulme
Copy link
Contributor

atoulme commented Apr 25, 2025

One change I ask is around choosing which attribute to map, and using both resource as well as metric, log, and span attributes.

Other than that, nice work!

I don't think that's needed. You'd use the groupbyattrs processor earlier in the pipeline to group your incoming data. This way you can make the data map to a nice payload.

@atoulme
Copy link
Contributor

atoulme commented Apr 25, 2025

Please address the failing CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

awss3exporter support mapping attributes
4 participants