Skip to content

Support for zstd compression in ingested files #57

@arthurzenika

Description

@arthurzenika

It's great that the plugin can decompress on the fly gzip files using the gzip_pattern https://www.elastic.co/docs/reference/logstash/plugins/plugins-inputs-s3#plugins-inputs-s3-gzip_pattern

It would be nice to have a similar feature for zstd compression that is being used by AWS in some use cases. We are using Lambda logs shipping to S3 as described in this tutorial https://docs.aws.amazon.com/lambda/latest/dg/logging-with-s3.html (through a loggroup with DELIVERY log_class) and the logs arrive compressed with zstd https://facebook.github.io/zstd/ On a linux host we can use unzstd from https://packages.debian.org/bookworm/zstd package to unzip it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions