[Enhancement] Support iceberg delete plan #67259
Conversation
|
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
[FE Incremental Coverage Report]✅ pass : 127 / 134 (94.78%) file detail
|
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
|
@cursor review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| execPlan.getFragments().get(0).setPipelineDop(1); | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Wrong sink flag set for Iceberg delete operations
The configurePipelineSink method unconditionally calls sinkFragment.setHasOlapTableSink() for all table types, including Iceberg tables. For Iceberg delete operations, this should call setHasIcebergTableSink() instead. The InsertPlanner correctly distinguishes between table types and sets the appropriate flag (setHasIcebergTableSink() for Iceberg tables). This mismatch could cause incorrect behavior in downstream code that checks which sink type is present using hasIcebergTableSink() vs hasOlapTableSink().
| * - _file (STRING): Path of the data file | ||
| * - _pos (BIGINT): Row position within the file | ||
| */ | ||
| public class IcebergDeleteSink extends DataSink { |
There was a problem hiding this comment.
Actually, you could make DeleteSink as a separate PR and then follow it with a delete plan PR.
4 participants
Why I'm doing:
#66944
What I'm doing:
This pull request adds support for DELETE operations on Iceberg tables in StarRocks. It introduces a new planning and execution path for Iceberg DELETEs, including a dedicated data sink, planner logic, and analyzer logic. The changes ensure that DELETE statements on Iceberg tables are properly transformed, validated, and executed, with appropriate support for partitioned tables and required metadata columns.
Key changes include:
Iceberg DELETE support in planner and execution:
IcebergDeleteSinkclass to handle delete operations for Iceberg tables, validating required columns and configuring the data sink for position deletes. (IcebergDeleteSink.java)DeletePlannerto detect Iceberg tables and set up the new sink, including logic to create the necessary tuple descriptor and handle physical property requirements (e.g., shuffle on partition columns).IcebergTableto include DELETE.Iceberg DELETE support in analyzer:
DeleteAnalyzerto rewrite Iceberg DELETE statements into a SELECT on the table's metadata columns (_file,_pos, and partition columns), and to enforce restrictions such as requiring a WHERE clause and disallowing partition or USING/CTE clauses.Test infrastructure updates:
_file,_pos) in Iceberg table schemas for unit testing.These changes collectively enable StarRocks to plan and execute DELETE statements on Iceberg tables using position deletes, with proper validation and partition-aware planning.
Fixes #issue
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check:
Note
Enables DELETE on Iceberg tables using position deletes with planner, analyzer, and sink support.
IcebergDeleteSinkwrites Parquet position delete files; validates_file(VARCHAR) and_pos(BIGINT); serializes via Thrift (ICEBERG_DELETE_SINK,TIcebergTableSinkwithdata_location)DeleteAnalyzerrewrites IcebergDELETEtoSELECT _file, _pos, [partition cols] FROM table WHERE ...; enforces requiredWHEREand disallows partitions/USING/CTEsDeletePlannerdetectsIcebergTable, builds shuffle property on partition columns, materializes required columns, and sets the new sinkIcebergTablenow advertisesDELETEingetSupportedOperations_file/_pos; minor test adjustmentsWritten by Cursor Bugbot for commit 7224bac. This will update automatically on new commits. Configure here.