Skip to content

awss3receiver: zstd compressed files (.binpb.zst) fail with "Unsupported file format" #47802

@mrkenkre

Description

@mrkenkre

Component(s)

receiver/awss3

What happened?

Description

When the awss3exporter is configured with compression: zstd, it writes files with a .zst suffix (e.g. metrics_123.binpb.zst). The awss3receiver correctly decompresses the data but fails to process it, logging:

Unsupported file format {"key": "metric/.../logs_123.binpb.zst"}

Steps to Reproduce

  1. Configure awss3exporter with compression: zstd and marshaler: otlp_proto
  2. Configure awss3receiver reading from the same bucket/prefix
  3. Observe "Unsupported file format" warnings for all .binpb.zst files

Expected Result

Files written by awss3exporter with compression: zstd are successfully decompressed and parsed by awss3receiver, with telemetry flowing through the pipeline without warnings.

Actual Result

awss3receiver logs "Unsupported file format" for every .binpb.zst file and silently drops all telemetry. No data reaches the downstream exporter.

Collector version

Confirmed on v0.150.0. Bug also present on main as of 2026-04-21.

Environment information

Environment

OS: RHEL 8 (Linux)
Compiler: N/A (using official container image)
Collector version: v0.150.1 (docker image: opentelemetry-collector-contrib:0.150.1)
S3-compatible endpoint: on-premises object storage (FlashBlade)

OpenTelemetry Collector configuration

Exporter (writes to S3):
  exporters:
    awss3:                                                                                                                                                                                                                                                                
      s3uploader:
        region: "us-east-1"                                                                                                                                                                                                                                               
        s3_bucket: 'otel'
        s3_prefix: 'metric' 
        compression: zstd                                                                                                                                                                                                                                                 
        s3_force_path_style: true                                                                                                                                                                                                                                         
      marshaler: otlp_proto                                                                                                                                                                                                                                               
                                                                                                                                                                                                                                                                          
  Receiver (reads from S3):
  receivers:                                                                                                                                                                                                                                                              
    awss3:  
      starttime: "2026-04-21 08:25"                                                                                                                                                                                                                                       
      endtime: "2026-04-21 08:30"  
      s3downloader:                                                                                                                                                                                                                                                       
        region: "us-east-1"
        s3_bucket: 'otel'                                                                                                                                                                                                                                              
        s3_prefix: 'metric' 
        s3_force_path_style: true

Log output

warn  awss3receiver@v0.150.0/receiver.go:276  Unsupported file format
{"otelcol.component.id": "awss3", "otelcol.component.kind": "receiver",                                                                                                                                                                                                 
  "otelcol.signal": "logs", "key": "metric/year=2026/month=04/day=20/hour=22/minute=14/logs_745183597.binpb.zst"}

Additional context

Switching the exporter to compression: gzip resolves the issue — files are read and processed correctly. The bug is isolated to the zstd code path in receiveBytes. The fix is a single missing line: key = strings.TrimSuffix(key, ".zst") mirroring the existing gzip handling.

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions