[WIP] Keep Iceberg implementation classes version-specific [fast-ut][reduced-it]

[gerashegalov]

Related to #15821.

Stacked on #15826.

Description

Finish the Iceberg jar-layout split by keeping all non-root-selected Iceberg implementation classes in the selected Spark shim directory instead of consolidating bitwise-identical copies into spark-shared.

Teach binary de-duplication that an explicitly root-selected class takes precedence over a matching keep-in-shim pattern. Add scoped Iceberg keep patterns, and remove the three remaining version-dependent Iceberg access helpers from dist/unshimmed-common-from-single-shim.txt.

The resulting layout keeps root-safe Iceberg APIs and helpers at the jar root, including the public catalog classes, while Spark- or Iceberg-version-dependent implementation classes remain in sparkXYZ/. No Iceberg-named classes remain in spark-shared.

AI assistance: Codex assisted with implementation and validation. The author reviewed the complete diff and this description before the PR was opened.

TODO before marking ready for review:

• Land the prerequisite draft PR [WIP] Unshim Iceberg reader and delete helpers [fast-ut][reduced-it] #15826.
• Refresh this stacked branch after the prerequisite lands.
• Confirm the complete multi-profile jar-layout audit in CI.
• Address CI and review feedback.

Validation:

• bash -n dist/scripts/binary-dedupe.sh
• Built matched Iceberg and aggregator artifacts for Spark 3.5.0, 3.5.7, 4.0.0, and 4.1.2.
• Ran direct parallel-world assembly and binary identity validation for Scala 2.12/Spark 3.5.0+3.5.7 and Scala 2.13/Spark 4.0.0+4.1.2.
• Verified both assembled layouts contain 41 Iceberg-named root classes and zero under spark-shared; the version-specific counts are 191 under spark350, 194 under spark357, 192 under spark400, and 197 under spark412.
• The direct assembly used --ignore-shim-revisions-check because cached non-Iceberg dependency artifacts carried different source revision metadata; assembly, binary identity validation, and the Iceberg layout audit completed successfully.

Performance: Not required because this changes class placement and classloader ownership only; it does not change runtime algorithms or executed operations.

Checklists

Documentation

• Updated for new or modified user-facing features or behaviors
• No user-facing change

Testing

• Added or modified tests to cover new code paths
• Covered by existing tests
  (Direct parallel-world packaging and binary identity validation for Spark 3.5.0, 3.5.7, 4.0.0, and 4.1.2.)
• Not required

Performance

• Tests ran and results are added in the PR description
• Issue filed with a link in the PR description
• Not required