-
Notifications
You must be signed in to change notification settings - Fork 132
Website: Add blog post for 25.0.0 #790
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+260
−0
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
dd538a5
Website: Add blog post for 25.0.0
raulcd f9f6812
Add C GLib and Ruby notes
kou 328e1ff
Add C++ and Parquet notes
pitrou ec2ae10
Also mention the hypot function
pitrou 4d69e13
Update Python notes
raulcd b91fd2b
Apply suggestions from code review
raulcd 3c254a9
Apply suggestion from @thisisnic
raulcd 1e2fd85
Fix headers, format and add missing Linux Packaging notes
raulcd File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,260 @@ | ||
| --- | ||
| layout: post | ||
| title: "Apache Arrow 25.0.0 Release" | ||
| date: "2026-07-10 00:00:00" | ||
| author: pmc | ||
| categories: [release] | ||
| --- | ||
| <!-- | ||
| {% comment %} | ||
| Licensed to the Apache Software Foundation (ASF) under one or more | ||
| contributor license agreements. See the NOTICE file distributed with | ||
| this work for additional information regarding copyright ownership. | ||
| The ASF licenses this file to you under the Apache License, Version 2.0 | ||
| (the "License"); you may not use this file except in compliance with | ||
| the License. You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| {% endcomment %} | ||
| --> | ||
|
|
||
| The Apache Arrow team is pleased to announce the 25.0.0 release. This release | ||
| covers over 3 months of development work and includes [**222 resolved | ||
| issues**][1] on [**268 distinct commits**][2] from [**66 distinct | ||
| contributors**][2]. See the [Install Page](https://arrow.apache.org/install/) to | ||
| learn how to get the libraries for your platform. | ||
|
|
||
| The release notes below are not exhaustive and only expose selected highlights | ||
| of the release. Many other bugfixes and improvements have been made: we refer | ||
| you to the [complete changelog][3]. | ||
|
|
||
|
|
||
| ## Format Notes | ||
|
|
||
| We clarified that variadic buffers exported over the C Data Interface could be | ||
| null ([GH-50255](https://github.com/apache/arrow/pull/50255)). Consumers of the C Data Interface must be ready to handle them. | ||
|
|
||
| ## Arrow Flight RPC Notes | ||
|
|
||
| The Flight SQL protocol was amended to let servers explicitly inform clients whether a prepared statement contains a result set or not ([GH-49497](https://github.com/apache/arrow/issues/49497)). | ||
|
|
||
| Progress was made on the ODBC driver for Flight SQL, but we are not yet distributing packages for end users at this time. | ||
|
|
||
| ## C++ Notes | ||
|
|
||
|
pitrou marked this conversation as resolved.
|
||
| ### CSV | ||
|
|
||
| A new option `default_column_type` disables type inference for all columns, including | ||
| those not listed in the `column_types` mapping ([GH-47663](https://github.com/apache/arrow/pull/47663)). | ||
|
|
||
| ### Compute | ||
|
|
||
|
raulcd marked this conversation as resolved.
|
||
| A new `hypot` compute function calculates Euclidean norms without the avoidable | ||
| overflow of a naive implementation | ||
| ([GH-50198](https://github.com/apache/arrow/pull/50198)). | ||
|
|
||
| Comparison functions have been upgraded to support StringView and BinaryView inputs | ||
| ([GH-49964](https://github.com/apache/arrow/pull/49964)). | ||
|
|
||
| Sort functions and their siblings (rank, select-k) now allow configuring per-key | ||
| null placement, so as to emulate SQL constructs such as | ||
| `ORDER BY i NULLS FIRST, j NULLS LAST` ([GH-46926](https://github.com/apache/arrow/pull/46926)). | ||
|
|
||
| Rank functions now correctly distinguish NaNs from null values in floating-point | ||
| arrays ([GH-45193](https://github.com/apache/arrow/pull/45193)). | ||
|
|
||
| The `count` function now accounts for logical nulls in run-end-encoded arrays ([GH-49908](https://github.com/apache/arrow/pull/49908)). | ||
|
|
||
| ### File Systems | ||
|
|
||
| The `FileSystemFactory` interface, used for dynamically-initialized filesystem | ||
| implementations, now allows passing a set of key-value pairs in addition to a | ||
| URI. This allows to pass sensitive initialization data, such as credentials, | ||
| without leaking them in the URI ([GH-50044](https://github.com/apache/arrow/pull/50044)). | ||
|
|
||
| ### IPC | ||
|
|
||
| We made the IPC reader stricter in a number of places, which could reject | ||
| invalid IPC streams or files that would previously appear to read successfully | ||
| (#49897, #50235). | ||
|
|
||
|
raulcd marked this conversation as resolved.
|
||
| ### Parquet | ||
|
|
||
| When writing a Parquet file with bloom filters enabled, bloom filters are | ||
| automatically "folded" so as to match the configured fpp (the max false positive | ||
| rate) according to the actual cardinality of the data used for the filter. | ||
| This can provide size savings, especially with the conservative default | ||
| cardinality estimate ([GH-50008](https://github.com/apache/arrow/pull/50008)). | ||
|
|
||
|
raulcd marked this conversation as resolved.
|
||
| Bloom filters can be faster on some platforms thanks to vectorization ([GH-50030](https://github.com/apache/arrow/pull/50030)). | ||
|
|
||
|
raulcd marked this conversation as resolved.
|
||
| It is now possible to read and write ListView data from/to Parquet ([GH-50160](https://github.com/apache/arrow/pull/50160)). | ||
|
|
||
| ### Miscellaneous C++ changes | ||
|
|
||
| On ARM64 platforms, Arrow C++ now supports dynamically dispatching to SVE-optimized | ||
| routines on compatible CPUs ([GH-49756](https://github.com/apache/arrow/pull/49756)). Previously, dynamic dispatch was only supported | ||
| on x86 platforms. | ||
|
|
||
| Runtime CPU detection now uses xsimd instead of home-grown detection functions | ||
| ([GH-49940](https://github.com/apache/arrow/pull/49940)). | ||
|
|
||
| The `ChunkedArray` class has a new method `ComputeLogicalNullCount`, mirroring | ||
| the existing methods of the same name on `Array` and `ArrayData` classes ([GH-50261](https://github.com/apache/arrow/pull/50261)). | ||
|
|
||
| The `Table` class has a new method `ToTensor` complementing the existing method | ||
| of the same on the `RecordBatch` class ([GH-41870](https://github.com/apache/arrow/pull/41870)). Both methods convert from the | ||
| columnar format to a contiguous two-dimensional array. | ||
|
|
||
|
raulcd marked this conversation as resolved.
|
||
| ## Linux Packaging Notes | ||
|
|
||
| The release dropped support for Debian bookworm [GH-50200](https://github.com/apache/arrow/issues/50200) due to | ||
| the distribution reaching End Of Life. | ||
|
|
||
| We added Reproducible Builds support for the Debian Linux Packages [GH-49988](https://github.com/apache/arrow/issues/49988). | ||
|
|
||
| ## Python Notes | ||
|
|
||
| ### Compatibility notes | ||
|
|
||
| * Feather reader and writer is deprecated in favour of the Arrow IPC API | ||
| [GH-49232](https://github.com/apache/arrow/issues/49232). | ||
|
|
||
| ### New features | ||
|
|
||
| * ``hypot`` compute kernel is added to Arrow compute module and accessible | ||
| in PyArrow [GH-50197](https://github.com/apache/arrow/issues/50197). | ||
| * ``pa.OSFile`` now accepts open file descriptor (``int`` parameter) besides | ||
| the ``str`` path [GH-49751](https://github.com/apache/arrow/issues/49751). | ||
| * Conversion from a list of individual ``numpy.ndarrays`` to a ``FixedShapeTensor`` | ||
| is added [GH-49644](https://github.com/apache/arrow/issues/49644). | ||
| * ``create_encryption_properties`` and ``create_decryption_properties`` methods | ||
| are added to the parquet API using Arrow C++ ``FileEncryptionPropertiesBuilder`` | ||
| and ``FileDecryptionPropertiesBuilder`` [GH-47435](https://github.com/apache/arrow/issues/47435). | ||
| * Conversion of Table to Tensor has been implemented in Arrow C++ and can also | ||
| be used in Python bindings [GH-40062](https://github.com/apache/arrow/issues/40062). | ||
| * ``default_column_type`` option is added to ``csv.ConvertOptions`` which sets a | ||
| default column type for all columns and can be combined with ``column_types`` | ||
| [GH-22232](https://github.com/apache/arrow/issues/22232). | ||
|
|
||
| ### Other improvements | ||
|
|
||
| * Extension types are supported in ``pyarrow.parquet.read_schema`` | ||
| [GH-48254](https://github.com/apache/arrow/issues/48254) | ||
| * Default values for Parquet `pre_buffer` are made consistent | ||
| [GH-49923](https://github.com/apache/arrow/issues/49923). | ||
|
|
||
| ### Relevant bug fixes | ||
|
|
||
| * ``count`` compute kernel bug for sliced union arrays is fixed | ||
| [GH-50113](https://github.com/apache/arrow/issues/50113). | ||
| * ``hash_any/hash_all`` compute kernel bug is fixed for sliced | ||
| boolean arrays [GH-50043](https://github.com/apache/arrow/issues/50043). | ||
| * ``Table.from_pylist`` on ``ExtensionType`` column with ``list_`` storage | ||
| crash when values exceed int32 offsets is fixed [GH-50012](https://github.com/apache/arrow/issues/50012). | ||
| * Bug causing Use-After-Free on ``PyList_SetItem`` in `SparseCSFTensorToNdarray` | ||
| is fixed [GH-49917](https://github.com/apache/arrow/issues/49917). | ||
| * Timezone drop when converting tz-aware pandas Categorical is fixed | ||
| [GH-49875](https://github.com/apache/arrow/issues/49875). | ||
| * `_export_to_c` segmentation fault for `binary_view` array is fixed by | ||
| fixing cast kernels so all-inline view arrays do not keep a null variadic | ||
| buffer slot [GH-49740](https://github.com/apache/arrow/issues/49740). | ||
| * `replace_with_mask` crash when null type inputs are used is | ||
| fixed [GH-47447](https://github.com/apache/arrow/issues/47447). | ||
| * Segmentation fault when using ``sort_indices`` for temporal types | ||
| is fixed [GH-47252](https://github.com/apache/arrow/issues/47252) | ||
| * Index level is bumped if pandas dataframe already contains ``__index_level_i__`` | ||
| column [GH-46179](https://github.com/apache/arrow/issues/46179). | ||
| * Special handling for single-file paths passed to ``ParquetDataset`` | ||
| constructor is restored, fixing merge error in ``pyarrow.parquet.read_table`` | ||
| [GH-43574](https://github.com/apache/arrow/issues/43574). | ||
|
|
||
| ### Other | ||
|
|
||
| * Annotations are withhold from Python wheels until they are complete | ||
| [GH-49831](https://github.com/apache/arrow/issues/49831). | ||
| * PyBuffer and NumPyBuffer destructors are protected against interpreter | ||
| finalization [GH-49942](https://github.com/apache/arrow/issues/49942). | ||
| * Documentation updates in [GH-50227](https://github.com/apache/arrow/issues/50227) | ||
| and [GH-20403](https://github.com/apache/arrow/issues/20403). | ||
| * Tests for regular ``replace_with_mask`` kernel usage are added | ||
| [GH-50072](https://github.com/apache/arrow/issues/50072). | ||
| * Hypothesis timezones test strategy now includes fixed offsets [GH-31318](https://github.com/apache/arrow/issues/31318). | ||
|
|
||
| ## R Notes | ||
|
|
||
|
raulcd marked this conversation as resolved.
|
||
| ### Breaking changes | ||
|
|
||
| - Arrow uint64 types are now always converted to R double (numeric) vectors, regardless of the values. Previously, small uint64 values were converted to R integer, which could cause inconsistent types within list columns when different list elements had different value ranges (#50339). | ||
|
|
||
| ### New features | ||
|
|
||
| - Field objects now support field-level metadata via $metadata and $with_metadata() (@max-romagnoli, #33390). | ||
| - Parquet files now support list-columns of ordered factors (ordered dictionaries) (#49689). | ||
|
|
||
| ### Minor improvements and fixes | ||
|
|
||
| - Array$create() now gives a clearer error message when given a POSIXct object with an invalid timezone (#40886). | ||
| - Dictionary arrays with large_string value types now convert correctly to R factors (#39603). | ||
| - open_dataset() now gives a clearer error message when providing a mix of readr and Arrow options (@Rich-T-kid, #33420). | ||
| - read_parquet() no longer triggers a C++ alignment warning from the Acero source node (#46178). | ||
| - Schema metadata partial matching on $metadata$r no longer errors when other metadata keys start with “r” (#50163). | ||
| - to_arrow() now preserves group_by() when converting from a dbplyr lazy table (#40640). | ||
| - write_parquet() now correctly validates that max_rows_per_group is a positive number (#40742). | ||
| - Stale S3 connections no longer cause a segfault during garbage collection (#50009). | ||
| - Spurious “Invalid metadata$r” warnings are no longer emitted when reading files with custom schema metadata (#48712). | ||
|
|
||
| ### Installation | ||
|
|
||
| - The R package now builds under r-universe/r-wasm (#49981). | ||
|
|
||
| ## Ruby and C GLib Notes | ||
|
|
||
|
kou marked this conversation as resolved.
|
||
| - Added fallback data type for unknown extension type: [GH-49969](https://github.com/apache/arrow/pull/49969) | ||
|
|
||
| ### Ruby | ||
|
|
||
| - Added `RecordBatch#merge`: [GH-50175](https://github.com/apache/arrow/pull/50175) | ||
| - Ensuring zero-initializing all `rb_memory_view_t` members for `rb_memory_view_get()`: [GH-50234](https://github.com/apache/arrow/pull/50234) | ||
|
|
||
| ### C GLib | ||
|
|
||
| No C GLib only notes. | ||
|
|
||
| ## Java, JavaScript, Go, .NET, Swift and Rust Notes | ||
|
|
||
| The Java, JavaScript, Go, .NET, Swift and Rust projects have moved to separate | ||
| repositories outside the main Arrow [monorepo](https://github.com/apache/arrow). | ||
|
|
||
| - For notes on the latest release of the [Java | ||
| implementation](https://github.com/apache/arrow-java), see the latest [Arrow | ||
| Java changelog][7]. | ||
| - For notes on the latest release of the [JavaScript | ||
| implementation](https://github.com/apache/arrow-js), see the latest [Arrow | ||
| JavaScript changelog][8]. | ||
| - For notes on the latest release of the [Rust | ||
| implementation](https://github.com/apache/arrow-rs) see the latest [Arrow Rust | ||
| changelog][5]. | ||
| - For notes on the latest release of the [Go | ||
| implementation](https://github.com/apache/arrow-go), see the latest [Arrow Go | ||
| changelog][6]. | ||
| - For notes on the latest release of the [.NET | ||
| implementation](https://github.com/apache/arrow-dotnet), see the latest [Arrow .NET changelog][9]. | ||
| - For notes on the latest release of the [Swift implementation](https://github.com/apache/arrow-swift), see the latest [Arrow Swift changelog][10]. | ||
|
|
||
| [1]: https://github.com/apache/arrow/milestone/74?closed=1 | ||
| [2]: {{ site.baseurl }}/release/25.0.0.html#contributors | ||
| [3]: {{ site.baseurl }}/release/25.0.0.html#changelog | ||
| [4]: {{ site.baseurl }}/docs/r/news/ | ||
| [5]: <https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md> | ||
| [6]: <https://github.com/apache/arrow-go/releases> | ||
| [7]: <https://github.com/apache/arrow-java/releases> | ||
| [8]: <https://github.com/apache/arrow-js/releases> | ||
| [9]: <https://github.com/apache/arrow-dotnet/releases> | ||
| [10]: <https://github.com/apache/arrow-swift/releases> | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.