Skip to content

Commit 6713c6a

Browse files
authored
Deprecate UCX-Py backends (#120)
Rewrite the UCXX/UCX-Py proxy backends added in #116 to deprecate UCX-Py. We have already removed the UCX-Py backend from Distributed in dask/distributed#9105, but while there's no release we cannot rely on that, so the changes in the PR work in the same way we expect when the migration is completed and Distributed has a new release. If distributed-ucxx is not installed a warning and exception are raised telling the user to install it. The state after this PR is as follows: 1. If distributed-ucxx is installed: 1. `protocol="ucx"` / `protocol="ucxx"` should continue working as before, both were already using distributed-ucxx 2. `protocol="ucx-old"` (only exists in rapids-dask-dependency added in #116) will raise the same `FutureWarning` as `protocol="ucx"` without RAPIDS 2. If distributed-ucxx is NOT installed: 1. `protocol="ucx"` / `protocol="ucxx"` / `protocol="ucx-old"` all raise a `FutureWarning` and don’t let users create processes. The final state is expected to be achieved when a new Distributed release is cut and the `__rdd_patch_ucx.py` file is removed from rapids-dask-dependency, which will be: 1. If distributed-ucxx is installed: 1. `protocol="ucx"` (now transparently uses distributed-ucxx) / `protocol="ucxx"` (always used distributed-ucxx) should continue working as before 2. If distributed-ucxx is NOT installed: 1. `protocol="ucx"` raises a `FutureWarning` and doesn’t let users create processes 2. `protocol="ucxx"` never existed before distributed-ucxx so it will just fail with a wrong protocol name Specifically I've tested the following to confirm what's behavior matches what is described above as the state of this PR (using all `ucx`/`ucxx`/`ucx-old` combinations): 1. `dask scheduler --protocol=ucx` 2. `dask worker ucx://127.0.0.1:8786` 3. `LocalCluster(protocol="ucx", silence_logs=False)` 4. `LocalCUDACluster(protocol="ucx", silence_logs=False)` Authors: - Peter Andreas Entschev (https://github.com/pentschev) Approvers: - Tom Augspurger (https://github.com/TomAugspurger) URL: #120
1 parent 0b15cc2 commit 6713c6a

File tree

1 file changed

+41
-621
lines changed

1 file changed

+41
-621
lines changed

0 commit comments

Comments
 (0)