Skip to content

fix(athena): properly escape identifiers in Iceberg SQL generation - #3423

Merged
kukushking merged 3 commits into
mainfrom
fix/iceberg-identifier-injection
Jul 31, 2026
Merged

fix(athena): properly escape identifiers in Iceberg SQL generation#3423
kukushking merged 3 commits into
mainfrom
fix/iceberg-identifier-injection

Conversation

@kukushking

@kukushking kukushking commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Description

Column and table identifiers are interpolated into the SQL that the Iceberg
write path generates (awswrangler/athena/_write_iceberg.py) without escaping.
Names containing the quoting character aren't quoted correctly, which produces
malformed SQL or unexpected behavior for identifiers that contain special
characters.

This can surface without any unusual caller input. During automatic schema
reconciliation (to_iceberg(..., fill_missing_columns_in_df=True), the
default), column names are read back from the AWS Glue Data Catalog and then
interpolated into the generated MERGE/INSERT. Glue permits identifiers that
the generation didn't handle safely, so this hardens the path against names
sourced from the catalog rather than only from the DataFrame.

Changes

Athena uses two identifier-quoting families depending on the statement:
Hive-style backticks for DDL (CREATE TABLE, ALTER TABLE) and Trino-style
double quotes for DML (SELECT/INSERT/MERGE/DELETE). This PR adds a
helper for each and applies it consistently:

  • _escape_athena_identifier() — doubles embedded ", for DML identifiers.
  • _escape_athena_ddl_identifier() — doubles embedded `, for DDL identifiers.

Applied to every identifier splice in the module:

  • DML: _merge_iceberg (MERGE/INSERT), delete_from_iceberg_table, and the
    mode="overwrite" DELETE FROM.
  • DDL: _create_iceberg_table (table + column names), _alter_iceberg_table_add_columns_sql,
    _alter_iceberg_table_change_columns_sql.
  • _build_order_by_clause column identifiers.
  • Routed the LOCATION '<path>' splice through the existing string-literal escaper.

Left unchanged on purpose: partition_cols are partition transform
expressions
(e.g. day(ts), truncate(10, col)), not plain identifiers, so
they are spliced verbatim rather than quoted.

Testing

  • Added unit tests for both helpers and each call site, using identifiers that
    contain the relevant quote character.
  • ruff format --check, ruff check, and mypy clean on the changed files.

@kukushking kukushking self-assigned this Jul 31, 2026

@EthanBunce EthanBunce left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@kukushking
kukushking merged commit 8a73ecb into main Jul 31, 2026
31 checks passed
@kukushking
kukushking deleted the fix/iceberg-identifier-injection branch July 31, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants