Skip to content

Commit 2e1eea7

Browse files
authored
chore: nanoarrow 0.9.0 release post (#799)
This PR adds a release post for nanoarrow 0.9.0!
1 parent 0f83405 commit 2e1eea7

1 file changed

Lines changed: 156 additions & 0 deletions

File tree

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
---
2+
layout: post
3+
title: "Apache Arrow nanoarrow 0.9.0 Release"
4+
date: "2026-08-14 00:00:00"
5+
author: pmc
6+
categories: [release]
7+
---
8+
<!--
9+
{% comment %}
10+
Licensed to the Apache Software Foundation (ASF) under one or more
11+
contributor license agreements. See the NOTICE file distributed with
12+
this work for additional information regarding copyright ownership.
13+
The ASF licenses this file to you under the Apache License, Version 2.0
14+
(the "License"); you may not use this file except in compliance with
15+
the License. You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.
24+
{% endcomment %}
25+
-->
26+
27+
The Apache Arrow team is pleased to announce the 0.9.0 release of
28+
Apache Arrow nanoarrow. This release consists of 38 resolved GitHub issues from
29+
5 contributors.
30+
31+
## Release Highlights
32+
33+
In addition to a number of bugfixes and minor build system improvements, we
34+
added several new features in nanoarrow 0.9.0.
35+
36+
- Dictionary decoding support in IPC reader
37+
- Reference-counted array/buffer support
38+
- LZ4 decompression support in R and Python bindings
39+
40+
See the
41+
[Changelog](https://github.com/apache/arrow-nanoarrow/blob/apache-arrow-nanoarrow-0.9.0/CHANGELOG.md)
42+
for a detailed list of contributions to this release.
43+
44+
## Features
45+
46+
### Dictionary decode support
47+
48+
Whereas the nanoarrow IPC reader suppports most Arrow IPC features, dictionary support
49+
was a long requested gap in the reader functionality (mostly requested by users of
50+
the [DuckDB nanoarrow extension](https://github.com/paleolimbot/duckdb-nanoarrow), which uses nanoarrow's reader). Dictionary encoding is used to reduce the size of frequently
51+
repeated values and is the serialized equivalent of the "dictionary" data type that
52+
is exposed in most Arrow implementations.
53+
54+
In nanoarrow 0.9.0 built with the IPC feature enabled, streams that include the most
55+
common forms of dictionary encoding (i.e., dictionary replacement) should now work
56+
out of the box. This includes nested/complex dictionary types and dictionary
57+
replacement but does not include "delta" dictionaries (i.e., dictionaires that
58+
grow larger as more values are encountered in the encoded values).
59+
60+
In R this is accessible via `read_nanoarrow()`; in Python this is accessible via
61+
`nanoarrow.ArrayStream.from_readable()`; in C this is available via the
62+
higher level `ArrowIpcArrayStreamReader` API. Lower level users of the
63+
`ArrowIpcDecoder` will have to update existing usage to use the
64+
`ArrowIpcDecoder...WithDictionaries()` variants of some functions
65+
to support input with dictionary schemas or batches.
66+
67+
### Reference-counted array/buffer support
68+
69+
In previous versions (since the introduction of the IPC reader), the
70+
`ArrowIpcSharedBuffer` has supported reading IPC streams and sharing an
71+
underlying set of data buffers for a group of arrays; however, this wasn't
72+
quite sufficient for the more complex case of decoding a dictionary and
73+
attaching cheaply-cloned shared "values" arrays for potentially many batches.
74+
Version 0.9.0 moves this functionality to the `ArrowSharedBuffer` and expands it
75+
to support moving all of an array's buffers into a shared state that can be
76+
more cheaply cloned.
77+
78+
### LZ4 decompression support in R and Python
79+
80+
While LZ4 decompression support was has long been available via the pluggable decoder
81+
framework (and available since 0.8.0 as a built-in compile time option), reading
82+
IPC streams with LZ4 buffer compression was not possible in the R or Python bindings.
83+
In 0.9.0, the requisite configuration options were added such that the packages are
84+
built with LZ4 when it is available on the system.
85+
86+
```python
87+
import io
88+
import nanoarrow as na
89+
import pyarrow as pa
90+
91+
buf = io.BytesIO()
92+
batch = pa.record_batch({"x": range(1000)})
93+
with pa.ipc.new_stream(buf, batch.schema, options=pa.ipc.IpcWriteOptions(compression="lz4")) as w:
94+
w.write_batch(batch)
95+
96+
buf.seek(0)
97+
na.ArrayStream.from_readable(buf).read_all()
98+
# nanoarrow.Array<non-nullable struct<x: int64>>[1000]
99+
# {'x': 0}
100+
# {'x': 1}
101+
# {'x': 2}
102+
# {'x': 3}
103+
# {'x': 4}
104+
# {'x': 5}
105+
# {'x': 6}
106+
# {'x': 7}
107+
# {'x': 8}
108+
# {'x': 9}
109+
# ...and 990 more items
110+
```
111+
112+
```r
113+
library(nanoarrow)
114+
library(reticulate)
115+
116+
# IPC Write with compression not available in arrow/R
117+
pa <- reticulate::import("pyarrow")
118+
io <- reticulate::import("io")
119+
120+
buf <- io$BytesIO()
121+
batch <- arrow::record_batch(x = 1:1000)
122+
writer <- pa$ipc$new_stream(buf, batch$schema, options = pa$ipc$IpcWriteOptions(compression = "lz4"))
123+
writer$write_batch(batch)
124+
writer$close()
125+
126+
nanoarrow::read_nanoarrow(as.raw(buf$getvalue())) |>
127+
tibble::as_tibble()
128+
#> # A tibble: 1,000 × 1
129+
#> x
130+
#> <int>
131+
#> 1 1
132+
#> 2 2
133+
#> 3 3
134+
#> 4 4
135+
#> 5 5
136+
#> 6 6
137+
#> 7 7
138+
#> 8 8
139+
#> 9 9
140+
#> 10 10
141+
#> # ℹ 990 more rows
142+
```
143+
144+
## Contributors
145+
146+
This release consists of contributions from 5 contributors in addition
147+
to the invaluable advice and support of the Apache Arrow community.
148+
149+
```console
150+
$ git shortlog -sn apache-arrow-nanoarrow-0.9.0.dev..apache-arrow-nanoarrow-0.9.0
151+
36 Dewey Dunnington
152+
2 Andrew Kane
153+
2 Bryce Mecum
154+
1 Michael Osipov
155+
1 Oliver Borchert
156+
```

0 commit comments

Comments
 (0)