Skip to content

bug: compression=auto does not work when the file has no extension. #18791

@wubx

Description

@wubx

Search before asking

  • I had searched in the issues and found no similar issues.

Version

main

What's Wrong?

create file format mygzcsv     TYPE = CSV,
    RECORD_DELIMITER = '\n',
    FIELD_DELIMITER = ',',
    COMPRESSION = AUTO,
    SKIP_HEADER = 1;

select $1,$2,$3,$4,$5,$6,$7,$8 from @mystage/gzcsv/01csvgz (file_format=>'mygzcsv');
error: APIError: QueryFailed: [1046]Invalid value '�0D�~���iC���"x�B���!`�b��z���<fw�59o��z����f��o����>�m��i���$����������d�Jj�E��^AWG�' for column 0 (_$1 String NULL): Invalid Utf8, cause: invalid utf-8 sequence of 1 bytes from index 0
at file 'gzcsv/01csvgz', line 1

create or replace file format mygzcsv     TYPE = CSV,
    RECORD_DELIMITER = '\n',
    FIELD_DELIMITER = ',',
    COMPRESSION = gzip,
    SKIP_HEADER = 1;

select $1,$2,$3,$4,$5,$6,$7,$8 from @mystage/gzcsv/01csv.gz (file_format=>'mygzcsv')
work is ok.

How to Reproduce?

any csv without extension name

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCategory: something isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions