GPU support

[ekwan]

Would xtb be accelerated using CUDA? That would be really nice for using xtb for dynamics. Thanks for making a great program!

[awvwgk]

xtb can already be accelerated using OpenACC (which is generating CUDA).

[awvwgk]

Try #407

[Chemistza74]

The compilation gets further, but I still get errors:

Build targets in project: 3

Found ninja-1.10.2 at /home/chris/miniconda3/bin/ninja
ninja: Entering directory `build'
[1/541] Compiling Fortran object lib_xtb.a.p/src_mctc_mctc_strings.f90.o
FAILED: lib_xtb.a.p/src_mctc_mctc_strings.f90.o
nvfortran -Ilib_xtb.a.p -I. -I.. -I../include -Minform=inform -O2 -g -fPIC -module lib_xtb.a.p -o lib_xtb.a.p/src_mctc_mctc_strings.f90.o -c ../src/mctc/mctc_strings.f90
NVFORTRAN-I-0035-Predefined intrinsic loc loses intrinsic property (../src/mctc/mctc_strings.f90: 343)
NVFORTRAN-S-0026-Unmatched quote (../src/mctc/mctc_strings.f90: 835)
0 inform, 0 warnings, 1 severes, 0 fatal for split
NVFORTRAN-S-0026-Unmatched quote (../src/mctc/mctc_strings.f90: 896)
0 inform, 0 warnings, 1 severes, 0 fatal for removebksl
[8/541] Compiling Fortran object lib_xtb.a.p/src_mctc_version.f90.o
NVFORTRAN-I-0035-Predefined intrinsic date loses intrinsic property (./xtb_version.fh: 2)
[11/541] Compiling Fortran object lib_xtb.a.p/src_mctc_chartools.f90.o
NVFORTRAN-I-0035-Predefined intrinsic ilen loses intrinsic property (../src/mctc/chartools.f90: 44)
[17/541] Compiling Fortran object lib_xtb.a.p/src_type_reader.F90.o
NVFORTRAN-I-0035-Predefined intrinsic size loses intrinsic property (../src/type/reader.F90: 131)
[20/541] Compiling Fortran object lib_xtb.a.p/src_mctc_systools.F90.o
NVFORTRAN-I-0035-Predefined intrinsic size loses intrinsic property (../src/mctc/systools.F90: 45)
[26/541] Compiling Fortran object lib_xtb.a.p/src_cqpath.f90.o
ninja: build stopped: subcommand failed.

Many thanks.

[awvwgk]

Looks like the -Mbackslash option is missing, try adding -Dfortran_args=-Mbackslash when setting up the build with meson.

[Chemistza74]

That did the trick! XTB now compiles and runs. However, the output I'm getting for a geometry optimisation is garbage:

........................................................................
.............................. CYCLE 1 ..............................
........................................................................
1 -62.4069560 -0.624070E+02 0.126E-06 9.76 0.0 T
2 -62.4069560 0.284217E-13 0.751E-07 9.76 20558.4 T
3 -62.4069560 0.710543E-14 0.168E-07 9.76 91839.1 T
SCC iter. ... 0 min, 0.012 sec
gradient ... 0 min, 0.022 sec

• total energy : -61.6755654 Eh change -0.2842171E-13 Eh
  gradient norm : NaN Eh/α predicted 0.0000000E+00 (-100.00%)
  displ. norm : NaN α lambda 0.0000000E+00
  maximum displ.: 0.0000000 α in ANC's #0, #0, #0, ...

........................................................................
.............................. CYCLE 2 ..............................
........................................................................
1 NaN NaN NaN NaN 0.0 T
2 NaN NaN NaN NaN 100000.0 T
3 NaN NaN NaN NaN 100000.0 T
4 NaN NaN NaN NaN 100000.0 T
5 NaN NaN NaN NaN 100000.0 T
6 NaN NaN NaN NaN 100000.0 T
7 NaN NaN NaN NaN 100000.0 T

It looks like it's calculating the correct energy, but gradient=NaN. Are there any other settings or options that I'm not including?

My build config looks like this:

export FC=nvfortran CC=nvc
meson setup build -Dla_backend=openblas -Dfortran_args=-Mbackslash -Dopenmp=false -Dgpu=true -Dcusolver=true -Dgpu_arch=86

I have a new RTX 3060TI GPU which I believe is gpu_arch=86 (I've also tried gpu_arch=70, with the same result).

Thanks again!

[awvwgk]

So far I used the GPU version only on Tesla V100 and Tesla K80 cards, so no experience with RTX cards up to now.

[awvwgk]

I added a short introduction to compiling xtb with NVHPC here:
https://xtb-docs.readthedocs.io/en/latest/development.html#building-with-gpu-support

[vsinha027]

Hi,
I am trying to compile xtb to be used on GPUs as well.
I followed the steps in the documentation.
The installation seemed to go fine.
But when I run the code I get the following error message:

xtb: symbol lookup error: /sw/arch/RedHatEnterpriseServer7/EB_production/2020/software/NVHPC/20.7/Linux_x86_64/20.7/compilers/lib/liblapack.so.0: undefined symbol: pgf90_str_copy_klen

Not sure how to handle this. Help/suggestions appreciated.

[awvwgk]

Looks like you might have not loaded the runtime libraries anymore in your environment.

[vsinha027]

the NVHPC module is loaded, I can also see it in the LD_LIBRARY_PATH
The xtb module is also loaded in the runtime.

Btw when I compiled I did it on the login node which is a CPU node. Could that be an issue?

Or are there more libraries to be loaded.

[vsinha027]

The GPUs are Tesla K40m, may be that is causing an issue.