v0.2.2

[jaychia]

Changes

• [CHORE] Edit 'make-hooks' command to install pre-commit script @colin-ho ([CHORE] Edit 'make-hooks' command to install pre-commit script #1602)
• [CHORE] Improve error messages when calling aggregation methods on dataframe without input columns @colin-ho ([CHORE] Improve error messages when calling aggregation methods on dataframe without input columns #1587)

✨ New Features

• [FEAT] Add translation of IOConfig to PyArrow filesystem arguments @jaychia ([FEAT] Add translation of IOConfig to PyArrow filesystem arguments #1592)
• [FEAT] [Scan Operator] Refactor planning and execution code to use shared Pushdowns struct. @clarkzinzow ([FEAT] [Scan Operator] Refactor planning and execution code to use shared Pushdowns struct. #1595)
• [FEAT] [Scan Operator] Add ChunkSpec for specifying format-specific per-file row subset selection for ScanTasks. @clarkzinzow ([FEAT] [Scan Operator] Add ChunkSpec for specifying format-specific per-file row subset selection for ScanTasks. #1590)
• [FEAT] [Scan Operator] Integrate size_bytes with ScanOperators @clarkzinzow ([FEAT] [Scan Operator] Integrate size_bytes with ScanOperators #1586)
• [FEAT] [Scan Operator] Add Python I/O support (+ JSON) to MicroPartition reads @clarkzinzow ([FEAT] [Scan Operator] Add Python I/O support (+ JSON) to MicroPartition reads #1578)
• [FEAT][ScanOperator 1/3] Add MVP e2e ScanOperator integration. @clarkzinzow ([FEAT][ScanOperator 1/3] Add MVP e2e ScanOperator integration. #1559)

🚀 Performance Improvements

• [PERF][REVERT] Reverts: use pyarrow table for pickling rather than ChunkedArray ([PERF] Use pyarrow table for pickling rather than ChunkedArray #1488) @jaychia ([PERF][REVERT] Reverts: use pyarrow table for pickling rather than ChunkedArray (#1488) #1605)
• [PERF] Speed Up MicroPartition Ops when we know the result is empty @samster25 ([PERF] Speed Up MicroPartition Ops when we know the result is empty #1604)

👾 Bug Fixes

• [BUG] clean up ray scheduler threads after computing partial results @samster25 ([BUG] clean up ray scheduler threads after computing partial results #1597)
• [BUG] Update requirements for typing_extensions @jaychia ([BUG] Update requirements for typing_extensions #1593)
• [BUG] Fix Deadlock with ScanOperators in to_physical_plan_scheduler and show iostats for glob and from_scan_task @samster25 ([BUG] Fix Deadlock with ScanOperators in to_physical_plan_scheduler and show iostats for glob and from_scan_task #1581)
• [BUG] add allow threads for io pool operations @samster25 ([BUG] add allow threads for io pool operations #1580)

🧰 Maintenance

• [CHORE] delete unused wheel tools @samster25 ([CHORE] delete unused wheel tools #1603)
• [CHORE] add IOStats to all micropartition ops @samster25 ([CHORE] add IOStats to all micropartition ops #1584)
• [CHORE] Use DAFT_MICROPARTITIONS as shared feature flag for data catalog support @jaychia ([CHORE] Use DAFT_MICROPARTITIONS as shared feature flag for data catalog support #1579)
• [CHORE] Convert GlobScanOperator to perform streaming into result and take a list of glob paths @jaychia ([CHORE] Convert GlobScanOperator to perform streaming into result and take a list of glob paths #1577)

⬆️ Dependencies

• Bump numpy from 1.25.2 to 1.26.2 @dependabot (Bump numpy from 1.25.2 to 1.26.2 #1596)

---

This discussion was created from the release v0.2.2.