Commit d24d6b3
authored
chore(deps): bump iceberg-rust (parallel file scanning) + standardize fork branch naming (spiceai#11328)
* chore(deps): bump iceberg-rust to parallel file scanning fork
Bumps the spiceai/iceberg-rust pin from e519b221 to b652de2e, picking up
the eager task-bucketing parallel-scan change (spiceai/iceberg-rust#42,
port of apache/iceberg-rust#2298).
IcebergTableProvider::scan() now plans files eagerly and distributes
FileScanTasks across min(target_partitions, n_files) DataFusion
partitions, so Iceberg file reads are scheduled concurrently instead of
through a single partition. Identity-partitioned tables additionally
declare Partitioning::Hash so downstream joins/aggregates can skip a
RepartitionExec.
No Spice code changes are required: Spice consumes the provider only via
IcebergTableProvider::try_new, and the change is internal to the scan
path. The fork preserves Spice's existing iceberg limit-pushdown.
* chore(deps): adopt spiceai-<iceberg>-df-<df> fork naming; re-pin to spiceai-0.9.1-df-53
Re-pins iceberg-rust to the renamed `spiceai-0.9.1-df-53` branch (rev
9e6e1a00) and bumps the version requirement to `=0.9.1` to match the
corrected crate version. Adds docs/dev/fork-branch-naming.md documenting
the `spiceai-<iceberg>-df-<datafusion>` convention.
The DF53 fork line was previously named `spiceai-0.9.0` but actually
tracks a post-0.9.1 `main` snapshot; the rename + version correction make
the branch name reflect its real contents.
* chore(deps): re-pin iceberg-rust to merged spiceai-0.9.1-df-53 commit
Fork PR spiceai/iceberg-rust#43 merged; re-pin from the branch head
(9e6e1a00) to the merge commit b8fc79d9 on spiceai-0.9.1-df-53.
* chore(deps): annotate iceberg pins with fork branch; drop naming doc
Address Copilot review on spiceai#11328: add `# branch: spiceai-0.9.1-df-53`
to the iceberg-rust pins (matching the convention used by
delta_kernel/duckdb/arrow). Remove docs/dev/fork-branch-naming.md per
review feedback (no separate doc).1 parent 79f8cb6 commit d24d6b3
2 files changed
Lines changed: 39 additions & 39 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
210 | 210 | | |
211 | 211 | | |
212 | 212 | | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
217 | | - | |
218 | | - | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
219 | 219 | | |
220 | 220 | | |
221 | 221 | | |
| |||
455 | 455 | | |
456 | 456 | | |
457 | 457 | | |
458 | | - | |
459 | | - | |
460 | | - | |
461 | | - | |
462 | | - | |
463 | | - | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
464 | 464 | | |
465 | 465 | | |
466 | 466 | | |
| |||
0 commit comments