Skip to content

Take fiona, rasterio and pyarrow from conda-forge - #2093

Open
MJohnson459 wants to merge 2 commits into
masterfrom
conda-fiona-rasterio
Open

MJohnson459 wants to merge 2 commits into
masterfrom
conda-fiona-rasterio

Conversation

@MJohnson459

@MJohnson459 MJohnson459 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

This is a minimal bump to dependencies to get more of the build from conda-forge. fiona, rasterio, and pyarrow are all available on conda-forge but the some shared dependency versions needed to be bumped to get this to work. This will also unlock the ability to bump and open up the versions of most of our dependencies.

PyPI published no linux-aarch64 wheel for fiona 1.10.1 or rasterio 1.4.3, so the aarch64 solve fell back to their sdists and compiled them. The prod environment has no compiler, so an arm64 prod install only worked when the build environment's wheels were already cached. The conda-forge builds cover all four platforms and link the same libgdal as the rest of the environment, instead of a second vendored GDAL.

conda-forge fiona 1.10.1 has no build against libgdal-core 3.11.1, and every conda rasterio 1.4.3 pins libgdal-core <3.11, so the geo stack moves together:

  gdal / libgdal-core  3.11.1 -> 3.11.5
  proj                 9.6.2  -> 9.7.1
  geos                 3.13.1 -> 3.14.1
  pyproj               3.7.1  -> 3.7.2
  shapely              2.1.1  -> 2.1.2
  pdal                 2.8.4  -> 2.9.2
  python-pdal          3.4.5  -> 3.5.3
  fiona                1.10.1 (pypi) -> 1.10.1 (conda)
  rasterio             1.4.3  (pypi) -> 1.4.4 (conda)

The pdal and gdal builds now fix libarrow at 22 on Linux, 21 on Windows and 20 on macOS. The PyPI pyarrow 20.0.0 pin existed to match conda's libarrow; it would no longer match on Windows, and no single version solves on every platform. pyarrow now comes from conda-forge, built against the environment's own libarrow, and the lock pins it per platform.

I tested this compiles on Linux and Windows.

Disclaimer - I did use AI to run testing here to find a suitable set of versions.

PyPI publishes no linux-aarch64 wheel for fiona 1.10.1 or rasterio
1.4.3, so the aarch64 solve fell back to their sdists and compiled them.
The prod environment has no compiler, so an arm64 prod install only
worked when the build environment's wheels were already cached. The
conda-forge builds cover all four platforms and link the same libgdal
as the rest of the environment, instead of a second vendored GDAL.

conda-forge fiona 1.10.1 has no build against libgdal-core 3.11.1, and
every conda rasterio 1.4.3 pins libgdal-core <3.11, so the geo stack
moves together:

  gdal / libgdal-core  3.11.1 -> 3.11.5
  proj                 9.6.2  -> 9.7.1
  geos                 3.13.1 -> 3.14.1
  pyproj               3.7.1  -> 3.7.2
  shapely              2.1.1  -> 2.1.2
  pdal                 2.8.4  -> 2.9.2
  python-pdal          3.4.5  -> 3.5.3
  fiona                1.10.1 (pypi) -> 1.10.1 (conda)
  rasterio             1.4.3  (pypi) -> 1.4.4 (conda)

The pdal and gdal builds now fix libarrow at 22 on Linux, 21 on Windows
and 20 on macOS. The PyPI pyarrow 20.0.0 pin existed to match conda's
libarrow; it would no longer match on Windows, and no single version
solves on every platform. pyarrow now comes from conda-forge, built
against the environment's own libarrow, and the lock pins it per
platform.

codem stays on PyPI; its rasterio requirement is satisfied by the conda
package. The only sdist left in any environment is vmem, which is pure
Python.

tests/test_gcp.py passes unchanged under PROJ 9.7.1.
pixi.lock is re-solved from the merged pixi.toml. It keeps the conda
geo stack from this branch and takes the headless py-opencv,
matplotlib-base, CPU onnxruntime and cuda-cudart changes from master.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant