Installation issue: llvm-amdgpu

[davidedelvento]

Steps to reproduce the issue

Building llvm-amdgpu requires an assembler able to understand zen4/zen5 VNNI instructions, otherwise an error such as Error: unsupported instruction 'vpdpwssd' happens at build time.

As far as I understand the GNU assembler provided with binutils v2.38+ will be sufficient to avoid this problem, however there is no binutils dependency in this package and if one is added it must be a binutils+gas (the defaults appears to be ~gas). I suspect the maintainers have new enough systems that the OS-provided binutils suffice, and therefore this problem is silently suppressed on their machines. I have v2.35 which caused this problem to occur.

Error message

Error message

Error: unsupported instruction 'vpdpwssd'

Information on your system

This is on official spack v1.1.1 release with whatever packages are automatically installed when cloning from scratch.

Additional information

@afzpatel @haampie @renjithravindrankannath @srekolam

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
• I have uploaded the build log and environment files
• I have searched the issues of this repo and believe this is not a duplicate

[yizeyi18]

Is it an issue of installing llvm-amdgpu, or all SIMD-enabled packages on zen4/zen5? Maybe target specific constraint is more helpful.

[haampie]

This isn't so much an llvm-amdgpu issue, but an issue with the system gcc toolchain.

You can spack install gcc to get a working compiler toolchain.

As for binutils, we could add it as a build dep, but I'm not sure if vpdpwssd was intended to be used, or just a choice of the compiler with optimizations for a certain target. If the former, then sure, if the latter, I don't think the llvm-admgpu package needs to fix it.

[davidedelvento]

Is it an issue of installing llvm-amdgpu, or all SIMD-enabled packages on zen4/zen5? Maybe target specific constraint is more helpful.

I don't know. This is the only one I experienced so far (and I've built plenty of other stuff, including e.g. OpenBLAS, Scalapack etc, but not yet BLIS and its friends).

You can spack install gcc to get a working compiler toolchain.

Yes, that was already what I did, and I was building with it not with OS-provided compiler

As for binutils, we could add it as a build dep, but I'm not sure if vpdpwssd was intended to be used, or just a choice of the compiler with optimizations for a certain target. If the former, then sure, if the latter, I don't think the llvm-admgpu package needs to fix it.

I don't know the answer and I see your point.

My opinion is slightly different than yours. Even if including vpdpwssd is a particular choice that happens only some circumstances, the fact is that binutils+gas is indeed a build dependency for llvm-amdgpu package and in my opinion it should be included regardless. Sure, one might use the OS-provided one and be done with it, but then I could argue the same with the compiler, no? If somebody really wants to do so they can always configure binutils as an external.
Moreover, I think we should give some weight to the compilation efforts: the binutils+gas build is very simple and small; and the llvm-amdgpu build is extremly long, CPU and memory consuming (in fact I spent hours and several OOMs before I could even reach the unsupported instruction error). As such, I would argue that the choice to add binutils (even if the OS-provided on could suffice) is a no brainer, since having that build failure is an absolute nightmare.

[haampie]

the fact is that binutils+gas is indeed a build dependency for llvm-amdgpu package

No, it's a dependency of gcc, and only gcc is a build dependency of llvm-amdgpu. Yes, llvm-amdgpu can put further constraints on that (for which it's easiest to make it a direct build dep and rely on unification of build dependencies).

Yes, that was already what I did, and I was building with it

Did you have e6d347d? (February 4th)

having that build failure is an absolute nightmare.

I agree. If this happened on RHEL though, I would first create an issue with them, it's so weird they have a broken toolchain for years now.

[davidedelvento]

No, it's a dependency of gcc, and only gcc is a build dependency of llvm-amdgpu. Yes, llvm-amdgpu can put further constraints on that (for which it's easiest to make it a direct build dep and rely on unification of build dependencies).

👍

Did you have e6d347d? (February 4th)

I was hoping that the separation of spack from spack-packages would have made my tracking of these things easier, but instead it made it harder (or I have not learned yet how to do it properly). It is particularly annoying that the spack-package repository is buried in a unwieldy location and that tags are placed on it very rarely, but I digress.

Anyway, given the February 4th timestamp of the commit and the fact that on my top-of-the-tree directory for the spack-package is February 2nd, we can assume the answer is no. This would most likely have avoided the issue (if I had it in place before building gcc, I guess)

Thanks!

[haampie]

Okay good. Up to you whether to keep this issue open. In the past I've fixed similar issues upstream where the package intended to use those instructions but failed to test both compiler and assembler: zlib-ng/zlib-ng#1562, zlib-ng/zlib-ng#1944.

In case of LLVM it's unclear if it's a package issue, you'd have to share the log.