Repository navigation
Take fiona, rasterio and pyarrow from conda-forge - #2093
Open
MJohnson459 wants to merge 2 commits into
Open
MJohnson459 wants to merge 2 commits into
MJohnson459 wants to merge 2 commits into
Conversation
PyPI publishes no linux-aarch64 wheel for fiona 1.10.1 or rasterio 1.4.3, so the aarch64 solve fell back to their sdists and compiled them. The prod environment has no compiler, so an arm64 prod install only worked when the build environment's wheels were already cached. The conda-forge builds cover all four platforms and link the same libgdal as the rest of the environment, instead of a second vendored GDAL. conda-forge fiona 1.10.1 has no build against libgdal-core 3.11.1, and every conda rasterio 1.4.3 pins libgdal-core <3.11, so the geo stack moves together: gdal / libgdal-core 3.11.1 -> 3.11.5 proj 9.6.2 -> 9.7.1 geos 3.13.1 -> 3.14.1 pyproj 3.7.1 -> 3.7.2 shapely 2.1.1 -> 2.1.2 pdal 2.8.4 -> 2.9.2 python-pdal 3.4.5 -> 3.5.3 fiona 1.10.1 (pypi) -> 1.10.1 (conda) rasterio 1.4.3 (pypi) -> 1.4.4 (conda) The pdal and gdal builds now fix libarrow at 22 on Linux, 21 on Windows and 20 on macOS. The PyPI pyarrow 20.0.0 pin existed to match conda's libarrow; it would no longer match on Windows, and no single version solves on every platform. pyarrow now comes from conda-forge, built against the environment's own libarrow, and the lock pins it per platform. codem stays on PyPI; its rasterio requirement is satisfied by the conda package. The only sdist left in any environment is vmem, which is pure Python. tests/test_gcp.py passes unchanged under PROJ 9.7.1.
pixi.lock is re-solved from the merged pixi.toml. It keeps the conda geo stack from this branch and takes the headless py-opencv, matplotlib-base, CPU onnxruntime and cuda-cudart changes from master.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a minimal bump to dependencies to get more of the build from conda-forge.
fiona,rasterio, andpyarroware all available on conda-forge but the some shared dependency versions needed to be bumped to get this to work. This will also unlock the ability to bump and open up the versions of most of our dependencies.PyPI published no linux-aarch64 wheel for fiona 1.10.1 or rasterio 1.4.3, so the aarch64 solve fell back to their sdists and compiled them. The prod environment has no compiler, so an arm64 prod install only worked when the build environment's wheels were already cached. The conda-forge builds cover all four platforms and link the same libgdal as the rest of the environment, instead of a second vendored GDAL.
conda-forge fiona 1.10.1 has no build against libgdal-core 3.11.1, and every conda rasterio 1.4.3 pins libgdal-core <3.11, so the geo stack moves together:
The pdal and gdal builds now fix libarrow at 22 on Linux, 21 on Windows and 20 on macOS. The PyPI pyarrow 20.0.0 pin existed to match conda's libarrow; it would no longer match on Windows, and no single version solves on every platform. pyarrow now comes from conda-forge, built against the environment's own libarrow, and the lock pins it per platform.
I tested this compiles on Linux and Windows.
Disclaimer - I did use AI to run testing here to find a suitable set of versions.