Skip to content

[extension/basicauthextension] Add AWS Secrets Manager support for basicauth extension #49025

Open
meebok wants to merge 18 commits into
open-telemetry:mainfrom
meebok:aws-secret-manager-support
Open

[extension/basicauthextension] Add AWS Secrets Manager support for basicauth extension #49025
meebok wants to merge 18 commits into
open-telemetry:mainfrom
meebok:aws-secret-manager-support

Conversation

@meebok

@meebok meebok commented Jun 11, 2026

Copy link
Copy Markdown

Description

  • Adds AWS Secrets Manager as a credential source for the basicauth extension, supporting both server (htpasswd) and client (username/password) modes with automatic background refresh.
  • Server mode: fetches raw htpasswd content from a Secrets Manager secret and refreshes it on a configurable interval
  • Client mode: fetches a JSON secret containing username/password keys and refreshes it on a configurable interval
  • Uses aws-sdk-go-v2/config.LoadDefaultConfig : workloads must have an IAM identity (instance profile, EKS pod identity/IRSA, ECS task role) with secretsmanager:GetSecretValue permission
  • Introduces internal/awssecretsmanager.Resolver : a callback-based fetcher with periodic refresh loop
  • Uses atomic.Pointer with CompareAndSwap for lock-free credential rotation during refresh

Link to tracking issue

Fixes #48277

Testing

  • Unit tests for Resolver (start, refresh, shutdown, error handling)
  • Unit tests for config validation (mutual exclusion, required fields)
  • Integration tests for both client and server AWS paths using mock SecretsManagerClient

Sample Configs

Server mode (validates inbound requests):

  extensions:                                                                                                                                                                                                                                       
    basicauth/server:
      htpasswd:                                                                                                                                                                                                                                     
        aws_secret:                                                                                                                                                                                                                                 
          secret_arn: "arn:aws:secretsmanager:us-east-1:000000000000:secret:test/server-htpasswd"                                                                                                                                                   
          region: "us-east-1"                                                                                                                                                                                                                       
          refresh_interval: 10s                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                    
  receivers:                                                                                                                                                                                                                                        
    otlp:                                                                                                                                                                                                                                           
      protocols:                                                                                                                                                                                                                                    
        http:                                                                                                                                                                                                                                       
          endpoint: 0.0.0.0:4318
          auth:                                                                                                                                                                                                                                     
            authenticator: basicauth/server                                                                                                                                                                                                         
                                                                                                                                                                                                                                                    
  exporters:                                                                                                                                                                                                                                        
    debug:                                                                                                                                                                                                                                          
      verbosity: detailed                                                                                                                                                                                                                           
                                                                                                                                                                                                                                                    
  service:                                                                                                                                                                                                                                          
    extensions: [basicauth/server]                                                                                                                                                                                                                  
    pipelines:                                                                                                                                                                                                                                      
      traces:                                                                                                                                                                                                                                       
        receivers: [otlp]                                                                                                                                                                                                                           
        exporters: [debug]                                                                                                                                                                                                                          

Client mode (attaches credentials to outbound requests):

extensions:                                                                                                                                                                                                                                       
  basicauth/client:                                                                                                                                                                                                                               
    client_auth:                                                                                                                                                                                                                                  
      aws_secret:                                                                                                                                                                                                                                 
        secret_arn: "arn:aws:secretsmanager:us-east-1:000000000000:secret:test/client-creds"                                                                                                                                                      
        region: "us-east-1"                                                                                                                                                                                                                       
        username_key: "user"                                                                                                                                                                                                                      
        password_key: "pass"                                                                                                                                                                                                                      
        refresh_interval: 10s                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                  
receivers:                                                                                                                                                                                                                                        
  otlp:                                                                                                                                                                                                                                           
    protocols:                                                                                                                                                                                                                                    
      http:                                                                                                                                                                                                                                       
        endpoint: 0.0.0.0:4319                                                                                                                                                                                                                    
                                                                                                                                                                                                                                                  
exporters:                                                                                                                                                                                                                                        
  otlphttp:     
    endpoint: http://localhost:4318                                                                                                                                                                                                               
    auth:                                                                                                                                                                                                                                         
      authenticator: basicauth/client                                                                                                                                                                                                             
                                                                                                                                                                                                                                                  
service:                                                                                                                                                                                                                                          
  extensions: [basicauth/client]                                                                                                                                                                                                                  
  pipelines:                                                                                                                                                                                                                                      
    traces:                                                                                                                                                                                                                                       
      receivers: [otlp]                                                                                                                                                                                                                           
      exporters: [otlphttp]                                                                                                                                                                                                                       

Collector Logs

Server collector startup + request handling:

2026-06-11T18:41:18.155-0400  info  extensions/extensions.go:45  Extension is starting...  {"otelcol.component.id": "basicauth/server"}                                                                                                           
2026-06-11T18:41:18.170-0400  info  extensions/extensions.go:62  Extension started.        {"otelcol.component.id": "basicauth/server"}                                                                                                           
2026-06-11T18:41:18.170-0400  info  otlpreceiver/otlp.go:175    Starting HTTP server       {"endpoint": "[::]:4318"}                                                                                                                              
2026-06-11T18:41:18.170-0400  info  service/service.go:264       Everything is ready. Begin running and processing data.                                                                                                                          

Client collector startup:

2026-06-11T18:41:21.851-0400  info  extensions/extensions.go:45  Extension is starting...  {"otelcol.component.id": "basicauth/client"}                                                                                                           
2026-06-11T18:41:21.865-0400  info  extensions/extensions.go:62  Extension started.        {"otelcol.component.id": "basicauth/client"}                                                                                                           
2026-06-11T18:41:21.865-0400  info  otlpreceiver/otlp.go:175    Starting HTTP server       {"endpoint": "[::]:4319"}                                                                                                                              
2026-06-11T18:41:21.865-0400  info  service/service.go:264       Everything is ready. Begin running and processing data.                                                                                                                          

Graceful shutdown (both):

2026-06-11T18:41:30.006-0400  info  service/service.go:278  Starting shutdown...                                                                                                                                                                  
2026-06-11T18:41:30.006-0400  info  extensions/extensions.go:69  Stopping extensions...                                                                                                                                                           
2026-06-11T18:41:30.006-0400  info  service/service.go:292  Shutdown complete.                                                                                                                                                                    

Prometheus Metrics

 Server collector (:8888/metrics):                                                                                                                                                                                                                 
 otelcol_receiver_accepted_spans{receiver="otlp",transport="http"} 1
 otelcol_receiver_refused_spans{receiver="otlp",transport="http"} 0                                                                                                                                                                                
 otelcol_exporter_sent_spans{exporter="debug"} 1                                                                                                                                                                                                   
                                                                                                                                                                                                                                                   
 Client collector (:8889/metrics):                                                                                                                                                                                                                 
 otelcol_receiver_accepted_spans{receiver="otlp",transport="http"} 1                                                                                                                                                                               
 otelcol_exporter_sent_spans{exporter="otlphttp",server_address="localhost",server_port="4318",url_path="/v1/traces"} 1                                                                                                                            
 otelcol_exporter_queue_size{data_type="traces",exporter="otlphttp"} 0   

Secret Rotation : Live Refresh Logs

The collector's background goroutine refreshes credentials every refresh_interval (10s here). When the secret is updated in AWS Secrets Manager, the new value is picked up atomically on the next tick . No restart required.

Client collector log (credentials rotated mid-flight):

  19:06:34.512  info  Extension is starting...        {"otelcol.component.id": "basicauth/client"}                                                                                                                                                  
  19:06:34.533  info  Extension started.              {"otelcol.component.id": "basicauth/client"}                                                                                                                                                  
  19:06:34.533  info  Everything is ready. Begin running and processing data.                                                                                                                                                                       
  19:06:44.546  info  secret refreshed successfully   {"secret_arn": "...test/client-creds"}  ← tick 1 (original creds)                                                                                                                             
  19:06:54.554  info  secret refreshed successfully   {"secret_arn": "...test/client-creds"}  ← tick 2 (rotated creds picked up)                                                                                                                    
  19:07:04.552  info  secret refreshed successfully   {"secret_arn": "...test/client-creds"}  ← tick 3                                                                                                                                              
  19:07:07.387  info  Received signal from OS         {"signal": "interrupt"}                                                                                                                                                                       
  19:07:07.388  info  Shutdown complete.           

Observations:

  • Refresh fires exactly every 10s (19:06:34 → 19:06:44 → 19:06:54 → 19:07:04)

Documentation

  • README updated with AWS Secrets Manager configuration options and IAM requirements

Authorship

  • I, a human, wrote this pull request description myself.

@meebok meebok requested a review from a team as a code owner June 11, 2026 20:28
@meebok meebok requested a review from andrzej-stencel June 11, 2026 20:28
@linux-foundation-easycla

linux-foundation-easycla Bot commented Jun 11, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

@github-actions

Copy link
Copy Markdown
Contributor

Welcome, contributor! Thank you for your contribution to opentelemetry-collector-contrib.

Important reminders:

  • Read our Contributing Guidelines.
  • Sign the CLA if you haven't already.
  • First-time contributors should have at most one PR not marked as draft until their first PR is merged.
  • If your change isn't one of our priority components, reviews may take more time.
  • Give reviewers at least a few days before pinging them for feedback.
  • If you need help or struggle to move your PR forward:

@github-actions github-actions Bot requested a review from frzifus June 11, 2026 20:28
meebok added 16 commits June 11, 2026 16:39
Add support for fetching credentials from AWS Secrets Manager as an
alternative credential source for both client and server authentication
modes. The resolver logic lives in a shared internal package at
extension/internal/awssecretsmanager/ and supports periodic background
refresh with atomic value updates for seamless secret rotation.
- Remove unused defaultRefreshInterval constant from config.go
- Remove requireKeys parameter from AWSSecretClientConfig.validate()
- Move logger field to end of basicAuthServer struct
- Restructure server Start() so AWS and file/inline paths are exclusive
- Pass *zap.Logger into newServerAuthExtension and newClientAuthExtension
  instead of assigning ext.logger after construction
- Simplify factory.go by removing intermediate variables
- Restore original inline-precedence comment from main
…esolver

Move the awssecretsmanager package from extension/internal/ to
basicauthextension/internal/ since no other extension needs it.

Add ClientResolver for client mode that fetches the secret once and
extracts both username and password in a single API call, replacing
the previous approach of two separate resolvers hitting the same
secret ARN independently.
Replace two nearly-identical types (Resolver + ClientResolver) with a
single Resolver that takes an onFetch(string) error callback. The caller
provides all parsing and storage logic in the callback, eliminating ~330
lines of duplicated lifecycle code.

Server mode callback rebuilds the htpasswd matcher atomically.
Client mode callback parses JSON and stores username+password via
CompareAndSwap.
@meebok meebok force-pushed the aws-secret-manager-support branch from a37ac8b to ac65fc4 Compare June 11, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[extension/basicauth] Add support for rotating credentials sourced from cloud secret managers

2 participants