alchemiops.py: Flag as unavailable on non-NVIDIA torch builds - #603
alchemiops.py: Flag as unavailable on non-NVIDIA torch builds#603yantar92 wants to merge 1 commit into
Conversation
|
Thanks for the contribution, I totally understand the issue but I am not sure this is the right fix because on CPU achemiops should be the default so the check needs to check if there is a non-cuda accelerator not just if there is not a cuda accelerator. |
* torch_sim/neighbors/alchemiops.py (_import_nvalchemiops_batch_neighbors):
Check that we are really using NVIDIA's drivers when checking if
nvalchemiops can be used with GPU. For CPU-only, nvalchemiops should
work.
* tests/test_neighbors.py (test_alchemiops_import_guard_non_nvidia_builds):
New test.
On ROCm builds, nvalchemiops correctly imports but does not work
during runtime, yielding
File "<...>/lib/python3.12/site-packages/torch_sim/neighbors/__init__.py", line 86, in torchsim_nl
return alchemiops_nl_n2(
^^^^^^^^^^^^^^^^^
File "<...>/lib/python3.12/site-packages/torch_sim/neighbors/alchemiops.py", line 62, in alchemiops_nl_n2
res = _batch_naive_neighbor_list(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<...>/lib/python3.12/site-packages/nvalchemiops/torch/neighbors/batch_naive.py", line 1513, in batch_naive_neighbor_list
compute_naive_num_shifts(cell, cutoff, pbc)
File "<...>/lib/python3.12/site-packages/nvalchemiops/torch/neighbors/neighbor_utils.py", line 326, in compute_naive_num_shifts
wp_device = wp.device_from_torch(device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<...>/lib/python3.12/site-packages/warp/_src/torch.py", line 39, in device_from_torch
return warp._src.context.runtime.cuda_devices[torch_device.index]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
IndexError: list index out of range
157e3cc to
6d81a4b
Compare
|
Oops. Did not realize that alchemiops works on CPUs. I have updated the commit. |
|
Btw, if you want to detect ROCm, you can use |
I have considered it, but AFAIU alchemiops will fail on any GPU that is not CUDA. |
|
I think that perhaps it's hard to do in logic for everyone's preferences so perhaps set 'TS_USE_ALCHEMIOPS=false' and skip on that is most direct approach and easy to document. If not set default to true. |
|
I am ok with that approach, but that would require more significant changes.
|
I don't think that adding another check for setting ALCHEMIOPS_AVAILABLE is a significant change it is less lines of code that this solution, indeed it could be a single line change without needing explicit tests imo. My opposition to this solution is that relying on 'torch.cuda.is_available()' to be true for a non-cuda device to get the logic tree to work for cpus is just very indirect and impossible to parse without knowing that amd just hides behind the cuda api and I would hope amd would avoid that obvious misdirection in future api updates.
Any brief note about the env variable would be surfaced instantly by an agent in debugging so I think that the env variable covers that documentation side. However if you're using TS on ROCm a lot it could also be cool to create a dedicated docs page called something like 'TorchSim on AMD' to capture all the learnings you have. |
Do I understand correctly that you are suggesting something like if not TS_USE_ALCHEMIOPS:
return Noneinside My concern here is that
Fair. What about the alternative proposed by @lil-lon with
I personally prefer to more direct approach, but you are the one to decide.
This is the first critical failure I encountered, really. Usually, on ROCm some NVIDIA-only optimizations are simply disabled (with a warning), but that does not crash the code. |
Oh, you meant environment variable. Then, my concern is indeed moot and the code change will be simple. |
Summary
On ROCm builds, nvalchemiops correctly imports but does not work during runtime, yielding
File "<...>/lib/python3.12/site-packages/torch_sim/neighbors/init.py", line 86, in torchsim_nl
return alchemiops_nl_n2(
^^^^^^^^^^^^^^^^^
File "<...>/lib/python3.12/site-packages/torch_sim/neighbors/alchemiops.py", line 62, in alchemiops_nl_n2
res = _batch_naive_neighbor_list(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<...>/lib/python3.12/site-packages/nvalchemiops/torch/neighbors/batch_naive.py", line 1513, in batch_naive_neighbor_list
compute_naive_num_shifts(cell, cutoff, pbc)
File "<...>/lib/python3.12/site-packages/nvalchemiops/torch/neighbors/neighbor_utils.py", line 326, in compute_naive_num_shifts
wp_device = wp.device_from_torch(device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<...>/lib/python3.12/site-packages/warp/_src/torch.py", line 39, in device_from_torch
return warp._src.context.runtime.cuda_devices[torch_device.index]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
IndexError: list index out of range
Checklist
Before a pull request can be merged, the following items must be checked:
_import_nvalchemiops_batch_neighbors, which I did not touch.