rebuild for ray V2.49.0, release python 3.13 limit by mattip · Pull Request #224 · conda-forge/ray-packages-feedstock

mattip · 2025-08-28T23:03:51Z

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

closes (again) #222

@apmorton I couldn't resist, feel free to close this if you have a similar PR.

mattip · 2025-08-28T23:04:25Z

@conda-forge-admin, please rerender

conda-forge-admin · 2025-08-28T23:05:19Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/17370084851. Examine the logs at this URL for more detail.}

mattip · 2025-08-28T23:06:18Z

recipe/meta.yaml

    # Avoid building openssl-1.1.1f and redis, not needed in the shipped package
    - patches/0011-do-not-build-redis-openssl.patch
+    # Ray patches grpc, update the patch for 1.67
+    - patches/0012-update-patch-for-grpc-1.67.patch


New patch, taken from ray-project/ray#51673. It is a shame that PR is stuck on one failing test file.

can you make this part of the vendor 1.67 patch rather than a second patch?

mattip · 2025-08-28T23:06:38Z

recipe/meta.yaml

    - patches/0003-remove-dependencies.patch
    # This needs modification for each release
-    - patches/0012-update-commit-sha.patch
-    - patches/0010-cython-shebang.patch  # [linux]


merged upstream, no longer needed

…5.08.28.20.53.1

mattip · 2025-08-29T04:43:01Z

macos builds are failing to find a compiler. Did the 'compilers' package change? In arm64 I see
bazel_toolchain/cc_wrapper.sh: line 57: arm64-apple-darwin20.0.0-clang: command not found

and similarly in x86_64 I see
bazel_toolchain/cc_wrapper.sh: line 57: x86_64-apple-darwin13.4.0-clang: command not found

mattip · 2025-08-29T06:49:23Z

Ahh, we can drop osx_64 (macos x86_64) builds, as upstream no longer supports macos x86_64. Is there talk on conda-forge about using arm64 macos build machines and not cross-compiling?

h-vetinari · 2025-08-29T06:55:39Z

Ahh, we can drop osx_64 (macos x86_64) builds, as upstream no longer supports macos x86_64.

Just because upstream drops them, we don't have to. There are very few differences between osx-64 and osx-arm64 (once you got the same deployment target), so that shouldn't be an issue. Though of course it can be dropped if there's an actual problem.

Is there talk on conda-forge about using arm64 macos build machines and not cross-compiling?

We're waiting for availability of osx-arm64 agents on azure pipelines, but osx-64 agents are still available on there for a while (e.g. macos-15 still supports it on AZP, even though it's been dropped for GHA).

mattip · 2025-08-29T07:50:45Z

The failure is at this line in the templated shell file
CONDA_BUILD_SYSROOT=${CONDA_BUILD_SYSROOT} ${CLANG} "$@". Note it is trying to call ${CLANG}, which for the x86_64 build is x86_64-apple-darwin13.4.0-clang. Locally I find this executable in the miniconda3/envs/ray-dev/conda-bld/ray-packages_1756452724796/_build_env/bin directory, and I can execute it. So maybe the problem is that the $PATH suddenly does not include the _build_env/bin somehow?

mattip · 2025-08-29T07:53:56Z

Ahh, indeed, the line before the error shows (locally)

  (cd /Users/mattip/miniconda3/envs/ray-dev/conda-bld/ray-packages_1756452724796/_build_env/share/bazel/6f0f404af16e17b93f84c222a50a19f3/sandbox/darwin-sandbox/4/execroot/io_ray && \
  exec env - \
    PATH=/bin:/usr/bin:/usr/local/bin \
    PWD=/proc/self/cwd \
  bazel_toolchain/cc_wrapper.sh

Note the $PATH is missing the conda env binary path.

Edit: so how did it work previously?
Edit2: indeed, if I build locally in a x86_64 conda env on main it all JustWorks.

apmorton · 2025-08-29T09:20:34Z

recipe/meta.yaml

    # Avoid building openssl-1.1.1f and redis, not needed in the shipped package
    - patches/0011-do-not-build-redis-openssl.patch
+    # Ray patches grpc, update the patch for 1.67
+    - patches/0012-update-patch-for-grpc-1.67.patch


can you make this part of the vendor 1.67 patch rather than a second patch?

apmorton · 2025-08-29T09:22:08Z

recipe/meta.yaml

        - msgpack-python >=1.0.0,<2.0.0
        - packaging
-        - protobuf >=3.15.3,!=3.19.5
+        - protobuf >=3.20.3


fwiw, I don't know when this happened, but it seems the following requirements are orphaned:

aiosignal frozenlist setproctitle

aiosignal and frozenlist were dropped in 2.45.0
`setproctitle is vendored in thirdparty/setproctitle from 2.48.0

apmorton · 2025-08-29T09:24:14Z

recipe/meta.yaml

  patches:
    - patches/0001-Disable-making-entry-scripts.patch
    - patches/0002-Ignore-warnings-in-event.cc-and-logging.cc.patch
    - patches/0003-remove-dependencies.patch


I don't think we need to patch out dependencies.

Instead of using setup.py install on unix, use:

"${PYTHON}" -m pip install . -vv --no-deps --no-build-isolation

(and add those flags to windows)

We already use -m pip install on windows, so this shouldn't be a problem.

Also, the cython build dep is out of date - it should be cython >=3.0.12.

If you switch to pip you'll also need to add it to host deps.

Does --no-deps hold for both extras_requires and install_requires kwargs in the setup_tools.setup() call?

apmorton · 2025-08-29T10:16:10Z

Linux builds fail in CI with:

ERROR: /home/conda/feedstock_root/build_artifacts/ray-packages_1756462154054/work/bazel/BUILD.bazel:15:11: //bazel:gen_extract depends on @py_deps_buildkite_bazel_runfiles//:pkg in repository @py_deps_buildkite_bazel_runfiles which failed to fetch. no such package '@py_deps_buildkite_bazel_runfiles//': whl_library py_deps_buildkite_bazel_runfiles failed:  (/home/conda/feedstock_root/build_artifacts/ray-packages_1756462154054/_build_env/share/bazel/2b025b9b96a5afec433c3ce619d9dfca/external/python3_9_x86_64-unknown-linux-gnu/bin/python3: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory

This was happening locally for me last night too, so at least I can reproduce.

mattip · 2025-08-29T13:08:13Z

Fixed the dependencies and the macOS build, still todo

collapse patch/0012 onto patch/0005
think about removing patch/0003 to not modify setup.py at all

mattip · 2025-08-29T13:09:42Z

Linux builds fail in CI with

Grr. Why are they installing such an old python? I will look at this later.

apmorton · 2025-08-29T18:06:56Z

macos CC_FOR_BUILD doesn't work for cross compile; attempting to disable bazel sandbox, since that seems to be what is blowing up the path...

mattip · 2025-08-31T06:12:04Z

macos CC_FOR_BUILD doesn't work for cross compile;

It worked locally. Where is the CI run with the commit 61e8c3c that tried this? I see various CI runs at https://dev.azure.com/conda-forge/feedstock-builds/_build?definitionId=11419&_a=summary. The commit sha in those runs is not the one in the branch. I see two CI runs: cd8441e4 (Merge e3bdcb1 into 526885e) and run d1a50fb34 (Merge 3874bd0 into 526885e). It seems something went wrong with azure pipelines, the run of 61e8c3c does not appear.

mattip · 2025-09-01T13:27:59Z

it might be worth a try to add a symlink

I have no sudo permissions inside the docker

mattip · 2025-09-01T13:29:21Z

its attempting to compile an arm64 assembly file as x86

That sounds familiar, I think we had this before.

mattip · 2025-09-01T13:55:35Z

Hmm. The previous problem led to this comment from the rules_boost maintainer:

With the continued caveat that I don't closely follow bazel targets/cross-compilation, the docs make it look to me like --cpu and --platforms are somewhat independent and they're in the middle of a migration from the one to the other. So you may need both?

conda-forge-admin · 2025-09-01T15:46:33Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found some lint.

Here's what I've got...

For recipe/recipe.yaml:

❌ Selectors in comment form no longer work in v1 recipes. Instead, if / then / else maps must be used. See lines [34].

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/17382187634. Examine the logs at this URL for more detail.}

mattip · 2025-09-01T15:47:29Z

Seems like its attempting to compile an arm64 assembly file as x86

Yup, cuz the CLANG=$CC_FOR_BUILD is using the wrong compiler, the cross-compilation has not been set up yet.

mattip · 2025-09-01T15:48:49Z

recipe/recipe.yaml

    # Avoid building openssl-1.1.1f and redis, not needed in the shipped package
    - patches/0011-do-not-build-redis-openssl.patch
+    # https://github.com/conda-forge/docker-images/issues/311
+    - patches/0012-use-unzip-from-env.patch  # [linux]


This needs conversion to if: then:

I'm experimenting with some fixes locally; I don't think we should be patching in this way - it should be possible to point to unzip installed to $PREFIX from conda-forge.

conda-forge-admin · 2025-09-01T16:25:23Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

mattip · 2025-09-01T16:27:49Z

recipe/recipe.yaml

        - psutil
        - python
        - setproctitle >=1.2.2,<1.4
+        - unzip


I think this needs to be m2-unzip on windows

mattip · 2025-09-02T12:00:15Z

Let's try to get this over the finish line without additional refactoring. There are people waiting for the release. Please do the boost refactoring after we have a successful merge of this PR.

apmorton · 2025-09-02T12:18:00Z

I specifically unvendored boost in hopes it would fix the osx_arm64 cross compilation builds - if you have an alternate fix I'm all ears.

apmorton · 2025-09-02T12:23:08Z

I bet we can drop the CC_FOR_BUILD stuff with recipe/patches/0012-No-path-isolation.patch actually if you'd like to revert the boost unvendoring and try that

mattip · 2025-09-02T12:42:00Z

I will try that change locally on an macsos-arm64 build. Thanks for finding the --incompatible_strict_action_env flag.

mattip · 2025-09-02T12:43:31Z

it would fix the osx_arm64 cross compilation builds

It turns out the CC_FOR_BUILD trick did not fix the problem: the wrong compiler was (is?) being used on osx_arm64

Edit: for future me, the rattler-build command requires target-platform:
rattler-build build --recipe ./recipe -m ./.ci_support/osx_arm64_python3.10.____cpython.yaml --test skip --target-platform osx-arm64

apmorton · 2025-09-02T13:57:07Z

Edit: for future me, the rattler-build command requires target-platform: rattler-build build --recipe ./recipe -m ./.ci_support/osx_arm64_python3.10.____cpython.yaml --test skip --target-platform osx-arm64

you should be able to use pixi run:

	build-linux_64_python3.13.____cp313
	build-locally
	build-osx_64_python3.10.____cpython
	build-osx_64_python3.13.____cp313
	build-osx_arm64_python3.13.____cp313
	build-win_64_python3.13.____cp313

there are also lint/rerender/smithy commands

This reverts commit e75d35a.

apmorton · 2025-09-02T16:52:15Z

recipe/build-core.sh

+  # Exchange the short exe name for the full path, used in
+  #   # bazel_toolchain/cc_wrapper.sh


stray comment now

Bleh. I will remove it in the upcoming 2.49.1 release, which should be happening in a few days

apmorton

LGTM once build is green

apmorton · 2025-09-02T20:36:36Z

Windows python 3.12 test is hung :(

python -c "import ray; ray.init('ray://127.0.0.1:10001')"

Given other windows versions passed, I'm going to proceed with merging in hopes its a fluke.
Worst case we're missing one windows version and debug in a follow up.

mattip · 2025-09-02T20:50:50Z

Thanks @apmorton

mattip added 3 commits August 29, 2025 00:45

update to ray 2.48.0 and redo patches

9597100

fix typos

65067d2

unskip python3.12

a6cdbf0

mattip requested review from apmorton, dHannasch, h-vetinari, krfricke, timkpaine and vnlitvinov as code owners August 28, 2025 23:03

mattip commented Aug 28, 2025

View reviewed changes

MNT: Re-rendered with conda-smithy 3.52.0 and conda-forge-pinning 202…

e3bdcb1

…5.08.28.20.53.1

apmorton reviewed Aug 29, 2025

View reviewed changes

rebuild

0c257a7

mattip added 2 commits August 29, 2025 16:07

use full path to compiler to fix problem with macos in bazel

958176a

from review: resync dependencies and use 'pip install --no-deps'

61e8c3c

apmorton added 2 commits August 29, 2025 11:05

linux building

3fb879e

need unzip, no nodejs variant, try no sandbox

3874bd0

mattip mentioned this pull request Sep 1, 2025

Update CODEOWNERS to remove krfricke #227

Closed

Merge remote-tracking branch 'origin/main' into v2.49.0

212d991

mattip commented Sep 1, 2025

View reviewed changes

Hopefully fix unzip and some other things

729f797

mattip commented Sep 1, 2025

View reviewed changes

Unvendor boost

e75d35a

mattip added 2 commits September 2, 2025 18:08

Revert "Unvendor boost"

ab53b25

This reverts commit e75d35a.

m2-zip on windows, remove hack for macos compilers

5270dbb

apmorton reviewed Sep 2, 2025

View reviewed changes

apmorton approved these changes Sep 2, 2025

View reviewed changes

mattip mentioned this pull request Sep 2, 2025

Redo boost unvendoring #229

Open

mattip linked an issue Sep 2, 2025 that may be closed by this pull request

Python 3.13 blockers? #218

Closed

mattip added the automerge Merge the PR when CI passes label Sep 2, 2025

apmorton merged commit d459f2e into conda-forge:main Sep 2, 2025
18 of 20 checks passed

mattip mentioned this pull request Sep 2, 2025

Unvendor boost #231

Open

5 tasks

mattip mentioned this pull request Nov 27, 2025

ray-packages v2.52.0 #250

Merged

5 tasks

		# Exchange the short exe name for the full path, used in
		# # bazel_toolchain/cc_wrapper.sh

Uh oh!

Conversation

mattip commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattip commented Aug 28, 2025

Uh oh!

conda-forge-admin commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattip commented Aug 29, 2025

Uh oh!

mattip commented Aug 29, 2025

Uh oh!

h-vetinari commented Aug 29, 2025

Uh oh!

mattip commented Aug 29, 2025

Uh oh!

mattip commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apmorton commented Aug 29, 2025

Uh oh!

mattip commented Aug 29, 2025

Uh oh!

mattip commented Aug 29, 2025

Uh oh!

apmorton commented Aug 29, 2025

Uh oh!

mattip commented Aug 31, 2025

Uh oh!

mattip commented Sep 1, 2025

Uh oh!

mattip commented Sep 1, 2025

Uh oh!

mattip commented Sep 1, 2025

Uh oh!

conda-forge-admin commented Sep 1, 2025

Uh oh!

mattip commented Sep 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

conda-forge-admin commented Sep 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattip commented Sep 2, 2025

Uh oh!

apmorton commented Sep 2, 2025

Uh oh!

apmorton commented Sep 2, 2025

Uh oh!

mattip commented Sep 2, 2025

Uh oh!

mattip commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apmorton commented Sep 2, 2025

Uh oh!

mattip commented Aug 28, 2025 •

edited

Loading

conda-forge-admin commented Aug 28, 2025 •

edited

Loading

mattip commented Aug 29, 2025 •

edited

Loading

mattip commented Sep 2, 2025 •

edited

Loading