0.60.0

0.60.0 largely involves unifying our vtable concepts as per the internal docs page in order to lay the groundwork for future extension points. Most notably: full Extension DType support, as well as AggregateFns.

Changes

⚠️ Breaks

• ScalarFn VTable (#6679) @gatesn
• Extension Scalar VTable (#6680) @connortsui20
• Rename spawn_blocking -> spawn_blocking_io (#6678) @gatesn
• Unify ExtVTable per vtable design doc (#6662) @gatesn
• Add an extension module to vortex (#6624) @connortsui20
• break: clean up compare (#6591) @joseph-isaacs
• Merge vortex-dtype into vortex-array (#6582) @connortsui20
• break: clean up compute (#6572) @joseph-isaacs
• Bump Rust MSRV to 1.90 (#6597) @robert3005
• break[array]: lazy list_contains (#6571) @joseph-isaacs

✨ Features

• Support building duckdb from a hash, including CI (#6658) @myrrc
• DuckDB to use the Scan API (#6652) @gatesn
• Use Scan API (#6391) @gatesn
• feat[gpu]: multi-stage dynamic dispatch with plan builder (#6636) @0ax1
• DuckDB lifetimes (#6620) @joseph-isaacs
• Expose table scan progress to DuckDB (#6596) @myrrc
• expose Executor, AbortHandle, AbortHandleRef, and Handle (#6595) @danking

🚀 Performance

• Avoid redundant chunk execution in pack_struct_chunks (#6682) @palaska
• read future to skip polled and dropped events for completed requests (#6667) @onursatici
• perf[gpu]: speed up dyn dispatch run-end kernel (#6665) @0ax1
• Route default append_to_builder through full execution loop (#6655) @palaska
• Add take-based rebuild strategy for ListView with small lists (#6492) @palaska
• oncelock shared array for zero cost access after materialisation (#6653) @onursatici
• oncelock shared array future on cuda flat layout (#6585) @onursatici
• cow buffers on array decode (#6590) @onursatici
• Extract to buffer once in list_contains (#6642) @palaska
• perf(layout): add empty-range fast paths in chunked reader (#6513) @caicancai
• chore[btrblocks]: more precise FoR expected_compression_ratio (#6301) @joseph-isaacs
• perf: remove double execute in exprs (#6608) @joseph-isaacs
• Use approx footer size for caching instead of walking the layout tree (#6601) @robert3005
• do not double validate the flatbuffer on the cuda layout (#6583) @onursatici
• increase default nvme coalescing (#6586) @onursatici
• decoding stats should lock once (#6587) @onursatici
• perf[array]: chunked take with bounded compute graph (#6589) @joseph-isaacs

🐛 Bug Fixes

14 changes

• cuda ctx bind to thread before unsafe (#6681) @onursatici
• keep DropOnSync alive until after dispatching for cuda slices (#6673) @onursatici
• Convert &mut to & for DuckDB GlobalData (#6634) @gatesn
• resize FSST compressor buffer to be large enough for largest string (#6676) @a10y
• Revert "Support building duckdb from a hash, including CI" (#6670) @myrrc
• Close DuckDB database before re-opening (#6654) @gatesn
• More minimal listview rebuild benches, force canonicalize (#6629) @palaska
• only a read lock for current_array_ref (#6631) @danking
• fix: faster codspeed bench for listview_rebuild (#6628) @joseph-isaacs
• Balance expression trees during binary stat falsification (#6622) @AdamGS
• Fix compression benchmarks build (#6614) @AdamGS
• Fix scalar deserialisation where value was unsigned/signed for signed/unsigned type (#6604) @robert3005
• fix: chunked array take with nulls (#6603) @joseph-isaacs
• fix: fuzzer repeat issue tracking (#6577) @joseph-isaacs

📖 Documentation

4 changes

• Language bindings docs (#6666) @gatesn
• Update docs for vtable unification plan (#6661) @gatesn
• docs: Remove unused CLI argument (code always uses temporary path) (#6637) @SYaoJun
• docs: clarify cuda async execution model (#6611) @0ax1

🧰 Maintenance

24 changes

• DuckDB: test curl impl, don't auto-load extensions (#6677) @myrrc
• chore: exclude cuda gen files from auto-formatting (#6671) @0ax1
• use match_each_canonical macro to reduce some canonical impl boilerplate (#6668) @AdamGS
• chore: trim unused crate deps with cargo-machete (#6663) @0ax1
• cudarc use 12.5 instead of 12.8 (#6660) @a10y
• docs: resurrect lost ddb build.rs comment (#6659) @0ax1
• chore[gpu]: alp + for + bp CUDA benchmark (#6644) @0ax1
• chore: clang-format - braces after if or for (#6643) @0ax1
• chore(deps): update codspeedhq/action digest to 2ac5728 (#6638) @renovate[bot]
• duckdb filesystem: don't use wrapper type (#6641) @myrrc
• use PublicBI fork with R2 hosted dataset (#6640) @a10y
• chore(deps): update all patch updates (#6639) @renovate[bot]
• Remove bad paths (#6626) @connortsui20
• Make the tui an default optional dependecy of vortex-python (#6625) @AdamGS
• refactor: simplify Buffer creation from Vec (#6621) @SYaoJun
• chore[ci]: explicitly use stable or night for all cargo commands (#6553) @joseph-isaacs
• Some nice to have things before extension types work (#6619) @connortsui20
• Use cache-cargo-install action instead of installing manually (#6616) @robert3005
• chore: delete commented out metadata tests (#6615) @vignesh-sankaran
• Add publish dry-run CI task (#6594) @robert3005
• More detailed backcompat comments (#6609) @connortsui20
• chore: rename device allocator to device read file (#6605) @0ax1
• chore: vortex-cuda only depend on vortex (#6602) @0ax1
• lower your expectations (#6579) @a10y