Skip to content

[exporter/file] Omit length-prefix framing for stream-decodable encodings when native compression is enabled#49384

Draft
paulojmdias wants to merge 6 commits into
open-telemetry:mainfrom
paulojmdias:fix/49328
Draft

[exporter/file] Omit length-prefix framing for stream-decodable encodings when native compression is enabled#49384
paulojmdias wants to merge 6 commits into
open-telemetry:mainfrom
paulojmdias:fix/49328

Conversation

@paulojmdias

@paulojmdias paulojmdias commented Jun 30, 2026

Copy link
Copy Markdown
Member

Description

Under the exporter.file.nativeCompression feature gate, output from a custom encoding extension was always length-prefixed (exportMessageAsBuffer), so text_encoding files decompressed to a 4-byte prefix + text instead of clean text. Now, encodings that are stream-decodable on read-back (implement encoding.LogsDecoderFactory, e.g., text_encoding) are written in newline-delimited format. Proto and other encodings keep length-prefix framing.

Link to tracking issue

Fixes #49328

Testing

Added unit tests: stream-decodable encoding yields clean newline-delimited output (no prefix), and verified end-to-end with text_encoding + zstd + the gate, and zstdcat output is now clean text.

Documentation

Updated native-compression note.

Authorship

  • I, a human, wrote this pull request description myself.

paulojmdias and others added 2 commits June 30, 2026 14:41
…ings when native compression is enabled, producing clean text output

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the fileexporter native-compression path to avoid 4-byte length-prefix framing for stream-decodable log encodings (e.g., text_encoding), so decompressed .zst output is clean, newline-delimited text suitable for standard tooling.

Changes:

  • Thread an encodingUnframed capability from the selected encoding extension into the file writer’s framing selection under exporter.file.nativeCompression.
  • Adjust native-compression framing selection to use newline-delimited output for eligible encodings (and keep length-prefix framing for proto/other encodings).
  • Add unit tests validating unframed/newline-delimited behavior for stream-decodable encodings, plus documentation and changelog updates.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
exporter/fileexporter/file_writer.go Adds encodingUnframed parameter to framing selection to choose newline-delimited output under native compression.
exporter/fileexporter/marshaller.go Detects stream-decodable encoding extensions (via encoding.LogsDecoderFactory) and records the capability.
exporter/fileexporter/file_exporter.go Passes encodingUnframed into framing selection for the standard exporter.
exporter/fileexporter/grouping_file_exporter.go Passes encodingUnframed into framing selection for the grouping exporter.
exporter/fileexporter/native_compression_test.go Adds tests validating unframed behavior for stream-decodable encodings and framing toggle behavior.
exporter/fileexporter/file_exporter_test.go Updates tests for the new buildExportFunc signature.
exporter/fileexporter/README.md Documents newline-delimited output for JSON and stream-decodable encodings under native compression.
exporter/fileexporter/go.mod Promotes extension/encoding to a direct dependency due to new imports.
.chloggen/fileexporter-unframed-text-encoding.yaml Changelog entry for the framing behavior change under the feature gate.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread exporter/fileexporter/file_writer.go
Comment thread exporter/fileexporter/native_compression_test.go
paulojmdias and others added 4 commits June 30, 2026 15:10
…-contrib into fix/49328

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>
…ector-contrib into fix/49328

Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>
Signed-off-by: Paulo Dias <paulodias.gm@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fileexporter: allow native zstd compression without length-prefix framing for text_encoding

2 participants