fix: merge upstream changes by heilhead · Pull Request #2 · WalletConnect/arrow-rs

heilhead · 2025-12-08T09:34:56Z

Updates the fork with the upstream changes.

Preview changes: apache/arrow-rs@main...WalletConnect:arrow-rs:fix/merge-upstream-081225

…ariant` kernel (apache#8282) # Which issue does this PR close? We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. - Closes apache#8281. # Rationale for this change # What changes are included in this PR? Support the last DataType in `cast_to_variant` kernel # Are these changes tested? Yes # Are there any user-facing changes? New type supported

# Which issue does this PR close? - Part of apache#8000 - Prep PR for apache#8340, to make it easier to review # Rationale for this change In apache#8340 I am trying to split the "IO" from the "where is the metadata in the file" from the "decode thrift into Rust structures" logic. I want to make it as easy as possible to review so I split it into pieces, but you can see apache#8340 for how it all fits together # What changes are included in this PR? This PR cleans up the code that handles parsing the 8 byte parquet file footer, `FooterTail`, into its own module and construtor # Are these changes tested? yes, by CI # Are there any user-facing changes? No, this is entirely internal reorganization and I left a `pub use` --------- Co-authored-by: Ed Seidl <etseidl@users.noreply.github.com> Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com>

…mp_nano (apache#8401) # Which issue does this PR close? - Closes apache#8331 . # Rationale for this change - Add typed_access for `Timestamp(Micro, _)` and `Timestamp(Nano, -)` # What changes are included in this PR? - Extract some data gen logic in tests to simplify the test logic (commit 93090d56717a6804e4862c23a0f85030b9f6406d), but it based on some old code(before apache#8392), rebase the master in the last commit - Add typed_access for `Timestamp(Micro, _)` and `Timestamp(Nano, _)` - Add test for typed_access for `Timestamp(Micro, _)` and `Timestamp(Nano, _)` # Are these changes tested? Covered by existing and added tests # Are there any user-facing changes? No

# Which issue does this PR close? \- # Rationale for this change `num` is a meta-crate that bundles functionality of a bunch of `num-*` crates, similar to how `futures` work: ```text num v0.4.3 ├── num-bigint v0.4.6 │ ├── num-integer v0.1.46 │ │ └── num-traits v0.2.19 (*) │ └── num-traits v0.2.19 (*) ├── num-complex v0.4.6 │ └── num-traits v0.2.19 (*) ├── num-integer v0.1.46 (*) ├── num-iter v0.1.45 │ ├── num-integer v0.1.46 (*) │ └── num-traits v0.2.19 (*) │ [build-dependencies] │ └── autocfg v1.5.0 ├── num-rational v0.4.2 │ ├── num-bigint v0.4.6 (*) │ ├── num-integer v0.1.46 (*) │ └── num-traits v0.2.19 (*) └── num-traits v0.2.19 (*) ``` We don't need all these sub-crates but only a very specific set. So instead of using the meta-crate, let's use the actual things we need. # What changes are included in this PR? Dependency changes. # Are these changes tested? It still compiles. # Are there any user-facing changes? Faster compilation.

# Which issue does this PR close? # Rationale for this change Field data type is useful when we try to downcast field builder. # What changes are included in this PR? Add `fields` getter method in `StructBuilder`. # Are these changes tested? CI. # Are there any user-facing changes? No.

# Which issue does this PR close? None. # Rationale for this change With apache#7836 complete we can update the release schedule. # What changes are included in this PR? Update the release schedule table in the README, removing released versions and adding three new releases. # Are these changes tested? No. # Are there any user-facing changes? No.

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-arith` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-cast` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-csv` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-flight` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-json` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-ipc` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

…pache#8462) # Which issue does this PR close? - Closes apache#8461 . # Rationale for this change See the issue # What changes are included in this PR? &mut -> & # Are these changes tested? Covered by existing # Are there any user-facing changes? No

# Which issue does this PR close? - Related to apache#8336 # Rationale for this change While working on apache#8438, I noticed that the enum variants of `ShreddingState` actually made the code (a lot) more complex than if I just referenced the (optional) value and typed_value columns directly. That made me wonder if `ShreddingState` would be better as a simple two-field struct. # What changes are included in this PR? Change `ShreddingState` to a two-field struct and update the few call sites that noticed. While we're at it, improve the docs about how shredding works. # Are these changes tested? Existing tests cover what is mostly an internal change # Are there any user-facing changes? `ShreddingState` is pub and changed from enum to struct. --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Fixes failing fmt job on main (https://github.com/apache/arrow-rs/actions/runs/18039752742/job/51335266375), caused by apache#8453.

…lType (apache#8408) # Which issue does this PR close? - Closes apache#8370 - Closes apache#8365 # Rationale for this change Parquet has logical types, which is how other writers signal what columns contain `Variant` values. # What changes are included in this PR? 1. Add mapping from Parquet LogicalType to/from Arrow ExtensionType added in apache#8392 2. Documentation and tests showing reading/writing Parquet files with the Variant logical annotation # Are these changes tested? Yes, new unit tests and doc examples # Are there any user-facing changes? You can now read/write Variant columns to `VariantArray` --------- Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com>

# Which issue does this PR close? None. # Rationale for this change Initial incremental builds are slower and their size is bigger, so we should disable them in CI. # What changes are included in this PR? Add `CARGO_INCREMENTAL=0` to CI env. # Are these changes tested? CI # Are there any user-facing changes? No

# Which issue does this PR close? None. # Rationale for this change https://blog.rust-lang.org/2025/09/18/Rust-1.90.0/ # What changes are included in this PR? - Bump rust version in `rust-toolchain.toml` to 1.90 - Fix new warnings - Rename `arrow-pyarrow-integration-testing/.cargo/config` to `config.toml` to fix deprecation warning # Are these changes tested? CI # Are there any user-facing changes? No

) # Which issue does this PR close? # Rationale for this change While changing the Thrift decoding more fine-grained benchmarking was desired. With that work mostly done, the benchmarks should revert to only testing public APIs. # What changes are included in this PR? Removes benches of internal APIs. Also makes changes to the "wide" test to be more representative of the real world. Use of random numbers added many negative values which require many more bytes to encode. # Are these changes tested? None required # Are there any user-facing changes? No, only changes to benchmark output.

# Which issue does this PR close? We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. - Closes apache#8411. # Rationale for this change The presence of relevant statistics when writing geometries and/or geographies is one of the primary motivations behind the GEOMETRY and/or GEOGRAPHY in Parquet. We'd like to make it easy for writers to provide them! # What changes are included in this PR? This PR introduces `Interval` and `WraparoundInterval` structs that handle interval math, and a `GeometryBounder` that iterates over input using the fantastic `wkb` crate (via `geo-traits`). # Are these changes tested? Yes! # Are there any user-facing changes? All public structures and functions are documented (although I am not sure what the final public API will be). --------- Co-authored-by: Kyle Barron <kylebarron2@gmail.com>

# Which issue does this PR close? - Builds on apache#7799 by supporting the new struct `GeospatialStatistics`. # Rationale for this change * This is part of a draft to support geospatial types (geometry and geography) in Parquet. This has been # What changes are included in this PR? - Structs for supporting geospatial statistics information (bbox and geospatial types) derived from thrift classes. - Would appreciate feedback on structure and where certain parts should go. # Are these changes tested? Yes # Are there any user-facing changes? If there are user-facing changes then we may require documentation to be updated before approving the PR. If there are any breaking changes to public APIs, please call them out.

# Which issue does this PR close? We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. - Part of apache#8474 # Rationale for this change The implications of using `Field::try_extension_type` may not be clear, so let's document it better. See apache#8474 for more background # What changes are included in this PR? Add some clarifying comments and a doc example Here is an example of it rendered <img width="1049" height="988" alt="Screenshot 2025-09-26 at 12 14 31 PM" src="https://github.com/user-attachments/assets/f7c8fc99-9204-4d9b-83ee-94ddd77abd02" /> # Are these changes tested? Yes new doc example tets # Are there any user-facing changes? No API / code changes

# Which issue does this PR close? - closes apache#5819 # Rationale for this change This allows writing generic code that accepts `&Field`, `Arc<Field>` or `&Arc<Field>`, or iterators of these. Same for `Schema`. # What changes are included in this PR? `impl AsRef<Field> for Field` `impl AsRef<Schema> for Schema` as suggested by `AsRef` own docs # Are these changes tested? yes # Are there any user-facing changes? it does not break API. Downstream crates cannot implement `AsRef<Field> for Field` Downstream crates cannot workaround these either - apache#5819 (comment)

# Which issue does this PR close? - Follow on to apache#8408 - Closes apache#7063 # Rationale for this change I was trying to consolidate the parquet extension type code after apache#8408, and in so doing I believe I actually found (and fixed) the root cause of apache#7063 (I will point it out inline) # What changes are included in this PR? 1. Consolidate parquet<-->arrow extension type metadata mapping in one module 2. Enable tests # Are these changes tested? Yes # Are there any user-facing changes? When reading parquet that is annotated with Json or UUID logical types, the resulting Arrow arrays will also have the canonical types attached.

…#7819) # Which issue does this PR close? We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. - Closes apache#7273 . # Rationale for this change Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. # What changes are included in this PR? There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. Added ryu and itoa to convert primitive numbers to strings # Are these changes tested? We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? I assume since I am not adding any functionality there are already tests covering this # Are there any user-facing changes? If there are user-facing changes then we may require documentation to be updated before approving the PR. If there are any breaking changes to public APIs, please call them out. There should not be Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-pyarrow-testing` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-ord` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-pyarrow` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

…he#8433) # Which issue does this PR close? - Part of apache#4886 # Rationale for this change This change adds support for Arrow DataType::Duration to the arrow-avro crate. Previous versions of this code return a NotYetImplemented error encouraging users to manually cast to Interval(MonthDayNano). With this change, Duration will now be represented as a long, with a logicalType value corresponding to the TimeUnit of the Arrow duration (`arrow.duration-nanos`, `arrow.duration-micros`, `arrow.duration-millis`, `arrow.duration-seconds`). This retains the Arrow paradigm for Duration of clock time, rather than the Avro approach of [calendar time](https://avro.apache.org/docs/1.11.1/specification/#duration). Because of this disconnect between the two specs, any attempt to map the value (e.g. seconds into days) would be lossy, as days and months can be of varying length. # What changes are included in this PR? Expands the arrow-avro crate to handle Arrow `Duration` types with various `TimeUnit` variants (Second, Millisecond, Microsecond, Nanosecond). Includes: - Additions in encoder/decoder to support Duration types. - Updates to schema handling for duration-specific metadata. - Comprehensive unit tests for reading and writing `Duration` types. # Are these changes tested? Yes, existing tests are all passing, and tests have been added to validate the encoding and decoding of Duration. A round trip test has been added to `writer/mod.rs`. # Are there any user-facing changes? - Crate is not yet public

# Which issue does this PR close? - Contribute to apache#6827 # Rationale for this change Splitting up apache#8227. # What changes are included in this PR? Migrate `arrow-pyarrow-integration-testing` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

…8909) # Which issue does this PR close? - Closes apache#8908. # Rationale for this change Add support for `ListView` and `LargeListView` in `ArrayData::new_null` # What changes are included in this PR? Handle match arm + add testcase # Are these changes tested? Yes # Are there any user-facing changes? No

# Which issue does this PR close? We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. - Closes apache#8910 . # Rationale for this change To fix the doc-test with feature-flag `prettyprint` # What changes are included in this PR? Enabled feature-flag `prettyprint` for specific doc-test that need this feature. # Are these changes tested? Yes # Are there any user-facing changes? No

# Which issue does this PR close? - Closes apache#8904. # Rationale for this change `GenericListViewArray::new_null` should return an array with correct len. # What changes are included in this PR? Construct zeroed offsets and sizes for list view. # Are these changes tested? Yes. # Are there any user-facing changes? No.

# Rationale for this change This PR adds convenience getter methods to the `UnionFields` api. It's very common to find a specific union field by `FieldRef` or type id

# Which issue does this PR close? This PR adds another method on the `UnionArray` api that returns a list of `FieldRef`s associated with the union type See: apache#8838 (comment)

# Which issue does this PR close? - Closes apache#8837 - Related to apache#8828 # Rationale for this change This PR implements comparison functionality for Union arrays. This implementation follows a simple ordering strategy where unions are first compared by their type identifier, and only when type identifiers match are the actual values within those types compared This approach handles both sparse and dense union modes correctly by using offsets when present (dense unions) or direct indices (sparse unions) to locate the appropriate child array values

# Which issue does this PR close? - Part of apache#8464 # Rationale for this change While releasing arrow 57.1.0 there is a new dependency on arrow-ord from arrow-select so I needed to change the order of publishing # What changes are included in this PR? Update instructions # Are these changes tested? I did so manually # Are there any user-facing changes? No

@alamb

# Which issue does this PR close? Closes apache#8870. # What changes are included in this PR? Check if field order in from/to casting matches, and if not, attempt to find the fields by name. # Are these changes tested? Added unit tests (that previously failed, so I separated them in a commit). # Are there any user-facing changes? No, it's strictly additive functionality. @alamb @vegarsti

@alamb

…ache#8914) # Which issue does this PR close? - Follow-up on apache#8901 # Rationale for this change It's non-obvious why the number "8" appears here. # What changes are included in this PR? Name the number such that it's more obvious that this is a conversion from bytes to bits. @alamb I can also include the [suggested comment](apache#8901 (comment)) if you prefer it. I thought the constant may have a lesser risk of becoming outdated without being noticed when changes to `MutableBuffer::new_null` happen. # Are these changes tested? - No behavior changes # Are there any user-facing changes? - No

@vustef

) # Which issue does this PR close? We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. - Related to https://github.com/apache/arrow-rs/pull/8874/files#r2550560291 # Rationale for this change @vustef [pointed](https://github.com/apache/arrow-rs/pull/8874/files#r2550560291) out the changelog for 57.1.0 improperly misses the row number support # What changes are included in this PR? Fix the Changelog NOTE this doesn't affect the changelog on the 57.1.0 tag -- I will however, add a note to apache#8800 to lead people there # Are these changes tested? CI # Are there any user-facing changes?

# Which issue does this PR close? - Closes apache#8926 . Note that creating sch arrays was already possible with the builder API (see `test_fixed_size_binary_builder_with_zero_value_length`) but not directly with the Array API. # Rationale for this change Avoid panicking (divide by zero) in `FixedSizeBinaryArray::try_new(0, ...)` # What changes are included in this PR? Special case for `size == 0` in `FixedSizeBinaryArray::try_new`. # Are these changes tested? Yes, additional constructor tests. # Are there any user-facing changes? Yes, no panics and the ability to directly create FSB arrays with zero-length items.

…valid `VariantArray` (apache#8921) # Which issue does this PR close? - part of apache#8887 # Rationale for this change In this comment here: apache#8887 (comment) One of the tests intended to cover the behavior of CastOptions actually contains an invalid Shredded variant (the values could not have been shredded successfully because the integer values are out of range) My understanding is that the test is intended to illustrate that casting fails when a non-shredded variant into a shredded variant. Thus, it would not have been possible to create the incorrect shredded variant in the first place (there would have been an error during shredding) # What changes are included in this PR? 1. Update the test to provide a valid VariantArray, and update the error message # Are these changes tested? Only tests # Are there any user-facing changes? None -- this is a test only change

# Which issue does this PR close? - Part of apache#8464 # Rationale for this change Variant and geospatial have been released, they now should follow the versioning of the other crates in this repo There seemed to be some issue publishing the 57.1.0 release with these crates. # What changes are included in this PR? Update versions to follow other crates # Are these changes tested? Yes by CI # Are there any user-facing changes? No

…pache#8887) This pr improves the performance of `variant_get` on a perfectly shredded variant, it bypasses the array builder and directly clone the shredded column. For example, if a variant looks like this: ``` optional group event (VARIANT) { required binary metadata; optional binary value; optional group typed_value { required group event_type { optional binary value; <- this is null optional binary typed_value (STRING); } } } ``` Then if we read `event_type` and we also want to cast it into a string, then we don't have to go through the builder but instead directly clone the `typed_value` array. Specifically this optimization is safe if: 1. `value` is null (does not exists) 2. `typed_value` has the same data type as the requested data type I think this is a pretty common case of variant shredding. ==== This PR also has benchmark code. It improves the performance by many many times (of course 😄). Let me know what you think! (fwiw, this pr is part of the efforts in datafusion-contrib/datafusion-variant#19 (comment)) --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Will panic if input buffer is invalid. **CLOUDFLARE ALERT!!!**

…metadata (apache#8797)

) # Which issue does this PR close? - Closes apache#8819 # Rationale for this change Adds support for using list view arrays through FFI # What changes are included in this PR? Support for the ListView types and arrays # Are these changes tested? I've added additional tests that verify round trips through FFI, and a pyarrow integration test. # Are there any user-facing changes? Extended support for ListView, doesn't effect existing functionality. --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

# Which issue does this PR close? We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. - Closes apache#8932 . # Rationale for this change Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. Some `panic!` conversions There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. # Are these changes tested? We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code Not sure these panic test cases are covered # Are there any user-facing changes? N/A

apache#8790) # Rationale for this change When dealing with Parquet files that have an exceedingly large amount of Binary or UTF8 data in one row group, there can be issues when returning a single RecordBatch because of index overflows (apache#7973). In `pyarrow` this is usually solved by representing data as a `pyarrow.Table` object whose columns are `ChunkedArray`s, which basically are just lists of Arrow Arrays, or alternatively, the `pyarrow.Table` is just a representation of a list of `RecordBatch`es. I'd like to build a function in PyO3 that returns a `pyarrow.Table`, very similar to [pyarrow's read_row_group method](https://arrow.apache.org/docs/python/generated/pyarrow.parquet.ParquetFile.html#pyarrow.parquet.ParquetFile.read_row_group). With that, we could have feature parity with `pyarrow` in circumstances of potential index overflows without resorting to type changes (such as reading the data as `LargeString` or `StringView` columns). Currently, AFAIS, there is no way in `arrow-pyarrow` to export a `pyarrow.Table` directly. Especially convenience methods from `Vec<RecordBatch>` seem to be missing. This PR tries to implement a convenience wrapper that allows directly exporting `pyarrow.Table`. # What changes are included in this PR? A new struct `Table` in the crate `arrow-pyarrow` is added which can be constructed from `Vec<RecordBatch>` or from `ArrowArrayStreamReader`. It implements `FromPyArrow` and `IntoPyArrow`. `FromPyArrow` will support anything that either implements the ArrowStreamReader protocol or is a RecordBatchReader, or has a `to_reader()` method which does that. `pyarrow.Table` does both of these things. `IntoPyArrow` will result int a `pyarrow.Table` on the Python side, constructed through `pyarrow.Table.from_batches(...)`. # Are these changes tested? Yes, in `arrow-pyarrow-integration-tests`. # Are there any user-facing changes? A new `Table` convience wrapper is added!

+    name: Audit
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+      - name: Install cargo-audit
+        run: cargo install cargo-audit
+      - name: Run audit check
+        run: cargo audit


+    name: Test
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
+      - name: Setup Rust toolchain
+        uses: ./.github/actions/setup-builder
+      - name: Test parquet-geospatial
+        run: cargo test -p parquet-geospatial
+
+  # test compilation
+  linux-features:


+    name: Check Compilation
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
+      - name: Setup Rust toolchain
+        uses: ./.github/actions/setup-builder
+      - name: Check compilation (parquet-geospatial)
+        run: cargo check -p parquet-geospatial
+
+  clippy:


+    name: Clippy
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+      - name: Setup Rust toolchain
+        uses: ./.github/actions/setup-builder
+      - name: Setup Clippy
+        run: rustup component add clippy
+      - name: Run clippy (parquet-geospatial)
+        run: cargo clippy -p parquet-geospatial --all-targets --all-features -- -D warnings


+    name: Test
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
+      - name: Setup Rust toolchain
+        uses: ./.github/actions/setup-builder
+      - name: Test parquet-variant
+        run: cargo test -p parquet-variant
+      - name: Test parquet-variant-json
+        run: cargo test -p parquet-variant-json
+      - name: Test parquet-variant-compute
+        run: cargo test -p parquet-variant-compute
+
+  # test compilation
+  linux-features:


+    name: Check Compilation
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          submodules: true
+      - name: Setup Rust toolchain
+        uses: ./.github/actions/setup-builder
+      - name: Check compilation (parquet-variant)
+        run: cargo check -p parquet-variant
+      - name: Check compilation (parquet-variant-json)
+        run: cargo check -p parquet-variant-json
+      - name: Check compilation (parquet-variant-compute)
+        run: cargo check -p parquet-variant-compute
+
+  clippy:


+    name: Clippy
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+      - name: Setup Rust toolchain
+        uses: ./.github/actions/setup-builder
+      - name: Setup Clippy
+        run: rustup component add clippy
+      - name: Run clippy (parquet-variant)
+        run: cargo clippy -p parquet-variant --all-targets --all-features -- -D warnings
+      - name: Run clippy (parquet-variant-json)
+        run: cargo clippy -p parquet-variant-json --all-targets --all-features -- -D warnings
+      - name: Run clippy (parquet-variant-compute)
+        run: cargo clippy -p parquet-variant-compute --all-targets --all-features -- -D warnings


+    name: Verify MSRV (Minimum Supported Rust Version)
+    runs-on: ubuntu-latest
+    container:
+      image: amd64/rust
+    steps:
+      - uses: actions/checkout@v6
+      - name: Setup Rust toolchain
+        uses: ./.github/actions/setup-builder
+      - name: Install cargo-msrv (if needed)
+        # cargo-msrv binary may be cached by the cargo cache step in setup-builder, and cargo install will error if it is already installed
+        run: if which cargo-msrv ; then echo "using existing cargo-msrv binary" ; else cargo install cargo-msrv ; fi
+      - name: Check all packages
+        run: |
+          # run `cargo msrv verify --manifest-path "path/to/Cargo.toml"` to see problematic dependencies
+          find . -mindepth 2 -name Cargo.toml | while read -r dir
+          do
+            echo "Checking package '$dir'"
+            cargo msrv verify --manifest-path "$dir" --output-format=json || exit 1
+          done


liamzwbao and others added 30 commits September 25, 2025 17:10

Fix Rustfmt in arrow-cast (apache#8473)

4d18401

Fixes failing fmt job on main (https://github.com/apache/arrow-rs/actions/runs/18039752742/job/51335266375), caused by apache#8453.

rambleraptor and others added 22 commits November 22, 2025 09:21

Add bit width check (apache#8888)

49ed6ef

Add getters to UnionFields (apache#8895)

558f744

# Rationale for this change This PR adds convenience getter methods to the `UnionFields` api. It's very common to find a specific union field by `FieldRef` or type id

Add UnionArray::fields (apache#8884)

5f3577a

# Which issue does this PR close? This PR adds another method on the `UnionArray` api that returns a list of `FieldRef`s associated with the union type See: apache#8838 (comment)

fix: ipc decode panic with invalid data (apache#8931)

b93fa52

Will panic if input buffer is invalid. **CLOUDFLARE ALERT!!!**

Add ability to skip or transform page encoding statistics in Parquet …

bab30ae

…metadata (apache#8797)

fix: merge upstream

8151e38

github-advanced-security AI found potential problems Dec 8, 2025

View reviewed changes

heilhead marked this pull request as ready for review December 8, 2025 09:36

heilhead requested review from chris13524, geekbrother, mario-reown and xDarksome December 8, 2025 09:36

mario-reown approved these changes Dec 8, 2025

View reviewed changes

heilhead merged commit c0d3f55 into master Dec 8, 2025
5 checks passed

Navigation Menu

Uh oh!

fix: merge upstream changes - #2

fix: merge upstream changes#2
heilhead merged 2142 commits into
masterfrom
fix/merge-upstream-081225

heilhead commented Dec 8, 2025 •

edited

Loading

Uh oh!

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Uh oh!

Conversation

heilhead commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Check warning

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

heilhead commented Dec 8, 2025 •

edited

Loading