Support full table scanning of partitioned table is prohibited

[melin]

Feature Request / Improvement

Many cloud customers expect spark sql to disable full table scanning by default, prompting users to add partition conditions.
If forget to bring the table partition condition, it will add a lot of cost

Query engine

None

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time

[RussellSpitzer]

This probably couldn't be done at the table format level since engines are the ones that decide what files they need to plan a query. This means you could do this but it would be a feature of Spark, Trino or whatnot.

[melin]

spark extensions: https://github.com/apache/kyuubi/blob/master/extensions/spark/kyuubi-extension-spark-3-5/src/main/scala/org/apache/kyuubi/sql/watchdog/MaxScanStrategy.scala

[ebyhr]

Trino already supports iceberg.query-partition-filter-required config property and query_partition_filter_required session property.