[destination-snowflake] Entire connection sync fails if one value is not recognized

[j-choe]

Connector Name

destination-snowflake

Connector Version

4.0.2

What step the error happened?

During the sync

Relevant information

Source: Postgres
Destination: Snowflake

While running a connection sync, if a single value in a row is not recognized, the entire sync fails. It doesn't seem like this row can be skipped or made generic by converting to a string. ON_ERROR cannot be configured because it's on the COPY INTO statement that Airbyte generates, not the snowflake stage.

Relevant log output

Warning from destination: Numeric value '{"acres": "66", "non_production_acres": "11.7"}' is not recognized File 'snowflake11038537174603430544.csv.gz', line 4090, character 0 Row 3221, column ""TABLENAMEcc5d4e637468e88a39d39f4196733fad""["PI":43] If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, please run 'info loading_data' in a SQL client.

net.snowflake.client.jdbc.SnowflakeSQLException: Numeric value '{"acres": "66", "non_production_acres": "11.7"}' is not recognized File 'snowflake11038537174603430544.csv.gz', line 4090, character 0 Row 3221, column ""TABLENAMEcc5d4e637468e88a39d39f4196733fad""["PI":43] If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, please run 'info loading_data' in a SQL client.

Contribute

• Yes, I want to contribute