I get this error when running the predict.py code, this is on an Ubuntu Linux 22.04 with an NVIDIA RTX A6000 GPU.
What could be the issue? Could it be that this is using the wrong version of some python or CUDA libraries?
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
$ dpkg -l nvidia-driver-535
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=================-===========================-============-=================================
ii nvidia-driver-535 535.129.03-0ubuntu0.22.04.1 amd64 NVIDIA driver metapackage
$ dpkg -l nvidia-cuda-toolkit
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-===============-============-=================================
ii nvidia-cuda-toolkit 11.5.1-1ubuntu1 amd64 NVIDIA CUDA development toolkit
Error:
N=1 L=491
Traceback (most recent call last):
File "/data2/cynicjon/RoseTTAFold2/network/predict.py", line 493, in <module>
pred.predict(
File "/data2/cynicjon/RoseTTAFold2/network/predict.py", line 318, in predict
self.run_prediction(
File "/data2/cynicjon/RoseTTAFold2/network/predict.py", line 392, in run_prediction
logit_s, logit_aa_s, _, logits_pae, p_bind, xyz_prev, alpha, symmsub, pred_lddt, msa_prev, pair_prev, state_prev = self.model(
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/data2/cynicjon/RoseTTAFold2/network/RoseTTAFoldModel.py", line 86, in forward
msa, pair, R, T, alpha, state, symmsub = self.simulator(
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/data2/cynicjon/RoseTTAFold2/network/Track_module.py", line 659, in forward
msa_full, pair, R_in, T_in, state, alpha, symmsub = self.extra_block[i_m](msa_full, pair,
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/data2/cynicjon/RoseTTAFold2/network/Track_module.py", line 562, in forward
R, T, state, alpha = self.str2str(
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
return func(*args, **kwargs)
File "/data2/cynicjon/RoseTTAFold2/network/Track_module.py", line 500, in forward
shift = self.se3(G, node, l1_feats, edge_feats)
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/data2/cynicjon/RoseTTAFold2/network/SE3_network.py", line 89, in forward
return self.se3(G, node_features, edge_features)
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/se3_transformer-1.0.0-py3.10.egg/se3_transformer/model/transformer.py", line 185, in forward
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/se3_transformer-1.0.0-py3.10.egg/se3_transformer/model/transformer.py", line 47, in forward
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/se3_transformer-1.0.0-py3.10.egg/se3_transformer/model/layers/attention.py", line 166, in forward
File "/home/petmedix/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/se3_transformer-1.0.0-py3.10.egg/se3_transformer/model/layers/convolution.py", line 319, in forward
File "/usr/local/lib/python3.10/dist-packages/dgl/view.py", line 179, in __call__
return self._graph.all_edges(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/dgl/heterograph.py", line 3589, in all_edges
src, dst, eid = self._graph.edges(self.get_etype_id(etype), order)
File "/usr/local/lib/python3.10/dist-packages/dgl/heterograph_index.py", line 696, in edges
edge_array = _CAPI_DGLHeteroEdges(self, int(etype), order)
File "dgl/_ffi/_cython/./function.pxi", line 295, in dgl._ffi._cy3.core.FunctionBase.__call__
File "dgl/_ffi/_cython/./function.pxi", line 227, in dgl._ffi._cy3.core.FuncCall
File "dgl/_ffi/_cython/./function.pxi", line 217, in dgl._ffi._cy3.core.FuncCall3
dgl._ffi.base.DGLError: [11:50:51] /opt/dgl/src/array/array.cc:42: Operator Range does not support cuda device.
Stack trace:
[bt] (0) /usr/local/lib/python3.10/dist-packages/dgl/libdgl.so(+0xf9f6f) [0x7f53c24f9f6f]
[bt] (1) /usr/local/lib/python3.10/dist-packages/dgl/libdgl.so(dgl::aten::Range(long, long, unsigned char, DGLContext)+0x117) [0x7f53c24fb1b7]
[bt] (2) /usr/local/lib/python3.10/dist-packages/dgl/libdgl.so(dgl::UnitGraph::COO::Edges(unsigned long, std::string const&) const+0x9b) [0x7f53c2a17bbb]
[bt] (3) /usr/local/lib/python3.10/dist-packages/dgl/libdgl.so(dgl::UnitGraph::Edges(unsigned long, std::string const&) const+0x9e) [0x7f53c2a1095e]
[bt] (4) /usr/local/lib/python3.10/dist-packages/dgl/libdgl.so(dgl::HeteroGraph::Edges(unsigned long, std::string const&) const+0x2a) [0x7f53c29112ca]
[bt] (5) /usr/local/lib/python3.10/dist-packages/dgl/libdgl.so(+0x51ab51) [0x7f53c291ab51]
[bt] (6) /usr/local/lib/python3.10/dist-packages/dgl/libdgl.so(DGLFuncCall+0x48) [0x7f53c28a9598]
[bt] (7) /usr/local/lib/python3.10/dist-packages/dgl/_ffi/_cy3/core.cpython-310-x86_64-linux-gnu.so(+0x15413) [0x7f53c1015413]
[bt] (8) /usr/local/lib/python3.10/dist-packages/dgl/_ffi/_cy3/core.cpython-310-x86_64-linux-gnu.so(+0x15c2b) [0x7f53c1015c2b]
I get this error when running the predict.py code, this is on an Ubuntu Linux 22.04 with an NVIDIA RTX A6000 GPU.
What could be the issue? Could it be that this is using the wrong version of some python or CUDA libraries?
Thanks in advance,