Skip to content

Source connectors do not honour file.compression.type #553

@Claudenw

Description

@Claudenw

The S3 and Azure source connectors do not decompress files that are gzipped or otherwise compressed.

The fix should be to add the decompress to the AbstractSourceRecordIterator to check if compression is set and use the appropriate decompress that is set in the configuration.

A test should be added to the integration tests to test that the compression is honoured and that when NONE is set that it is also honoured.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions