Clamp or error on oversize values written to Iceberg per out_of_range_values

[sfc-gh-mslot]

Summary

• Three opt-in GUCs (pg_lake_engine.iceberg_max_string_bytes, iceberg_max_binary_bytes, iceberg_max_nested_type_bytes; default 0 = disabled) bound the byte size of values written to Iceberg tables.
• Behavior on an oversize value follows the table's out_of_range_values option — same contract as the existing temporal / numeric OOR clamp:
  • 'error' (default): raise an error identifying the column / type / byte size / exceeded GUC.
  • 'clamp': silently fix up the value (truncate text/bytea, NULL jsonb/json, NULL the whole container for arrays/composites/maps).
• Both the per-tuple FDW write path and the SQL pushdown wrapper (around the SELECT used by INSERT..SELECT, COPY FROM, and postgres_scan) honor the policy and produce identical output under 'clamp'.

Motivation

Some downstream consumers of Iceberg tables impose per-column byte caps smaller than what PostgreSQL values in the source can carry. For example, on Snowflake the column-type byte ceilings are:

• STRING / VARCHAR: 16 MiB default, up to 128 MiB when declared with an explicit larger length.
• BINARY: 8 MiB default, up to 64 MiB.
• OBJECT / ARRAY / VARIANT: 128 MiB.

Without a guard, rows whose individual values exceed the target column's cap reach the consumer and surface as opaque "value too long" errors when the consumer ingests them. Operators set the GUCs to whatever the destination column's actual ceiling is.

Behavior

PG type	Action under 'clamp'
text / varchar / bpchar	truncate at UTF-8 character boundary
bytea	byte-truncate
jsonb / json	replace with NULL (truncating would yield invalid JSON)
array / composite / map	NULL the whole container if its measured byte size exceeds the nested-type limit

Under 'error' (default), the same conditions raise instead of fixing up the value. Error messages identify the column, the source type, the actual byte size, the exceeded GUC, and a hint pointing at out_of_range_values = 'clamp'.

For jsonb, the limit applies to the text-serialized form (what the consumer sees). For arrays/composites the cap is approximated by the varlena content size — Snowflake stores semi-structured data without per-record headers, so on-disk size is a close-enough proxy for the JSON length the consumer will see.

Implementation

• Per-tuple FDW path (pg_lake_table.c): IcebergSizeCheckOrClampSlotInPlace dispatches on the table's outOfRangePolicy and passes the column name into IcebergSizeClampDatum so the error message can name it.
• SQL pushdown path (iceberg_query_validation.c): IcebergWrapQueryWithSizeClampChecks takes the policy and emits either the existing clamp UDFs (iceberg_size_clamp_text / _blob, iceberg_byte_size-driven CASE for nested) or new check UDFs (iceberg_size_check_text / _blob, error() inside CASE for jsonb / nested) which raise on oversize.
• DuckDB UDFs (duckdb_pglake/src/duckdb_pglake_extension.cpp): two new iceberg_size_check_* scalar functions throw InvalidInputException with the column name and exceeded GUC.

Test plan

• pg_lake_table/tests/pytests/test_iceberg_size_clamping.py — 13 cases covering both modes (10 clamp + 3 error-default), both paths (per-tuple FDW + SQL pushdown), and all clampable types (text/varchar/bpchar/bytea/jsonb/json/array/composite/int[]/composite-of-bigint).
• Existing regressions: test_special_numeric.py and test_iceberg_validation.py still pass — the new code lives alongside IcebergErrorOrClampDatum and re-uses its FDW slot-level entry point.
• CI confirmation.

🤖 Generated with Claude Code