xTB gpu_build problems

[larsakth]

Hello,
I have tried to build xtb 6.4.1 from source for gpu's (Nvidia V100) on a machine running under CentOS 7 following the online instructions using:

• nvfortran 21.7
• meson 0.59.0
• ninja 1.10.2
  (TeraChem and Amber have been installedand run on the gpu's, and thus there is nothing inherently wrong with the machine)

meson was run with the following instructions (no problems):
$ meson setup build_gpu --prefix=$HOME/.local -Dla_backend=netlib -Dgpu=true -Dcusolver=true -Dgpu_arch=70

However, the build fails when running:
$ ninja -C build_gpu install

ninja stops with very minimal error messages:
ninja: build stopped: subcommand failed.
It can be re-run a few times compiling about 10 subroutines at a time before stopping until it stops permanently.

Any suggestions on how to move forward?

Thanks,
Lars

[awvwgk]

Thanks for the report, an actual output showing the error would certainly be helpful.

[awvwgk]

Thanks that is more helpful.

The issue originates from the OpenMP support in the NVHPC compilers, 21.7 should support OpenMP 4.5 and parts of OpenMP 5.0 already. Following OpenMP 4.5 the usage of simd directives in pure procedures is valid:

OpenMP directives except SIMD and declare target directives may not appear in PURE or ELEMENTAL procedures.

Seems like a bug in NVHPC 21.7 to me, I'll put together a reproducer and submit it to NVIDIA at some point.

---

To move this forward, we probably have to work around this compiler bug to get xtb working on GPUs again. Not sure when I will be able to spend a couple of days on this kind of work (again).

[larsakth]

I understand. Compiler bugs are always tricky to work around. My experience from cpu-based compilers is that changing compiler version can work around some bugs. Worth for me to try a slightly older version of nvhpc?

[awvwgk]

I don't think this will work here. You could try to compile an older version of xtb (6.4.0) which doesn't use the simd in pure procedures yet.

[larsakth]

Indeed, v.6.4.0 seems to compile without error messages. I will make some tests.

[larsakth]

Using a sloppy and fast input file, the compiled v.6.4.0 version runs on one gpu card. The calculation gave the impression to be running rather heavily also on the cpu. I might be back later with some questions regarding gpu card control etc. Thanks so far!!

[mstevens-bcs]

Has there been any progress on a resolution here?

[pwging13]

Also interested in the current status of GPU support. Is this same behavior being observed for 6.5.1 with the latest NVHPC SDK?