[pkg/ottl] Add client metadata access to OTTL contexts

[ronkorland]

Description

This PR adds metadata access capability to all OTTL (OpenTelemetry Transformation Language) contexts, enabling users to access client request metadata within OTTL transformations.

What this achieves:

• Adds new metadata path expression to access the entire client metadata object
• Adds metadata["key"] path expression to access specific metadata values by key
• Available in all OTTL contexts (span, log, metric, resource, datapoint, scope, spanevent, profile, profilesample)
• Enables access to HTTP headers, authentication tokens, and other client metadata from incoming requests
• Supports filtering, routing, and enrichment use cases based on request metadata

Implementation details:

• Created new ctxmetadata package with path expression parser
• Integrated metadata access into the common OTTL context parser
• Returns client.Metadata type for full metadata access
• Returns string or nil for specific key access (first value if multiple exist)
• Read-only access (metadata cannot be modified via OTTL)

Link to tracking issue

Fixes #33288

Testing

• Added comprehensive unit tests in pkg/ottl/contexts/internal/ctxmetadata/metadata_test.go
• Tests cover accessing entire metadata object and specific keys
• Tests validate behavior with empty keys, non-existent keys, and multi-value scenarios
• Tests verify error handling for invalid access patterns
• All existing OTTL context tests continue to pass

Documentation

• Updated README files for all OTTL contexts to document the new metadata paths:
  • pkg/ottl/contexts/ottldatapoint/README.md
  • pkg/ottl/contexts/ottllog/README.md
  • pkg/ottl/contexts/ottlmetric/README.md
  • pkg/ottl/contexts/ottlprofile/README.md
  • pkg/ottl/contexts/ottlprofilesample/README.md
  • pkg/ottl/contexts/ottlresource/README.md
  • pkg/ottl/contexts/ottlscope/README.md
  • pkg/ottl/contexts/ottlspan/README.md
  • pkg/ottl/contexts/ottlspanevent/README.md
• Each README now includes:
  • metadata path description and return type (client.Metadata)
  • metadata[""] path description and return type (string or nil)
  • Clear explanation of behavior for missing keys and multiple values

[edmocosta]

Thanks for the help with the renaming and conflict resolution, @edmocosta! I've reviewed the changes and they look good. Please let me know if there's anything else needed on my end to move this forward

Thanks @ronkorland, I've addressed the final suggestions I had, one adapting the transform context to use pointer (recent standard changes), added some security and best practices section on the docs, and finally a feature gate to temporarily allow users to disable the new context access, see 3685ffa and 50be0be.
Could you please take a look on both commits?

@TylerHelmuth @evan-bradley, it looks good to me, please have a final look so we can finally move this feature forward. Thank you!

[ronkorland]

@edmocosta thanks for the ping. I have reviewed the latest changes in 3685ffa and 50be0be. The security documentation and the feature gate are excellent additions. I'm aligned with these updates and ready for the code owners' final review

[edmocosta]

/easycla

[evan-bradley]

Thanks @ronkorland. Looks good to me.

[ronkorland]

@evan-bradley I've addressed your feedback:
Feature Gate: Added the reference URL and version (v0.147.0).
Refactoring: Cleaned up the otelcol path handling and context logic.

[evan-bradley]

Thanks @ronkorland. Looks good to me, just needs a make gotidy.

[ronkorland]

Thanks @ronkorland. Looks good to me, just needs a make gotidy.

done

[otelbot]

Thank you for your contribution @ronkorland! 🎉 We would like to hear from you about your experience contributing to OpenTelemetry by taking a few minutes to fill out this survey. If you are getting started contributing, you can also join the CNCF Slack channel #opentelemetry-new-contributors to ask for guidance and get help.

[evan-bradley]

Saw again that this was opened in August of last year, just want to thank you again for being patient throughout the review process.

[edmocosta]

Thank you @ronkorland for your great work and patience. I'm glad this feature finally landed :)

[ronkorland]

Thank you @edmocosta and @evan-bradley for the thorough reviews and for sticking with this PR since August! I really appreciate the guidance on the feature gate implementation and the final refactoring.

I'm excited to see this land, and I'll be keeping an eye on the tracking issue for the feature gate moving forward. I'd be happy to jump back in for any further code changes or improvements needed, so please don't hesitate to reach out