Skip to content

[WIP] Unshim Iceberg reader and delete helpers [fast-ut][reduced-it] - #15826

Draft
gerashegalov wants to merge 9 commits into
NVIDIA:mainfrom
gerashegalov:codex/unshim-iceberg-delete-filter-helpers
Draft

[WIP] Unshim Iceberg reader and delete helpers [fast-ut][reduced-it]#15826
gerashegalov wants to merge 9 commits into
NVIDIA:mainfrom
gerashegalov:codex/unshim-iceberg-delete-filter-helpers

Conversation

@gerashegalov

@gerashegalov gerashegalov commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Related to #15821.

Stacked on #15824.

Description

Continue reducing special root-layout treatment for Iceberg by grouping several small, closely related helper refactors:

  • Move position-delete field IDs, reader mode types, reader utility methods, and the deletion-vector container into the root-safe iceberg-common module.
  • Move delete-filter constants and helper methods into the existing version-specific GpuDeleteFileInfo companion.
  • Remove six obsolete Iceberg class entries from dist/unshimmed-common-from-single-shim.txt.

The logic is unchanged. Root-safe helpers are now emitted directly at the jar root, while helpers tied to Iceberg implementation classes remain in the selected parallel world.

AI assistance: Codex assisted with implementation and validation. The author reviewed the complete diff and this description before the PR was updated.

TODO before marking ready for review:

Validation:

  • Built iceberg-common for Spark 3.3.0, 3.5.7, 4.0.0, and 4.1.2.
  • Built the Iceberg 1.6, 1.9, and 1.10 consumer modules.
  • Ran GpuDeleteFilterSuite: 6 tests passed.
  • Ran GpuPostProcessorSuite: 18 tests passed.
  • Ran IcebergDeletionVectorSuite: 5 tests passed.
  • Verified the new helpers are present in the root-safe module artifacts and the removed companions are absent from the version artifacts.

Performance: Not required because this change only relocates helpers or changes their owning companion without changing algorithms, data flow, or executed operations.

Checklists

Documentation

  • Updated for new or modified user-facing features or behaviors
  • No user-facing change

Testing

  • Added or modified tests to cover new code paths
  • Covered by existing tests
    (GpuDeleteFilterSuite, GpuPostProcessorSuite, and IcebergDeletionVectorSuite.)
  • Not required

Performance

  • Tests ran and results are added in the PR description
  • Issue filed with a link in the PR description
  • Not required

Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
@gerashegalov gerashegalov changed the title [WIP] Make Iceberg delete-filter helpers version-specific [fast-ut][reduced-it] [WIP] Unshim Iceberg reader and delete helpers [fast-ut][reduced-it] Aug 29, 2026
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants