[Feature Request]: Support a `--vector-library` compilation flag for Chapel

[jabraham17]

In a recent investigation of the performance of Chapel vs Fortran (especially Intel Fortran), I found that by enabling clang/LLVM's auto vectorization with a vector library Chapel code could be further vectorized, especially Math library routines like log2 and pow (which normally are not vectorizable).

This is achievable today in Chapel through some heroics, but only using the C backend.

chpl --fast --no-ieee-float --ccflags -fveclib=...

Note that --no-ieee-float is required, otherwise due to floating point precision constraints vectorization will be limited. I am also focusing here on -fveclib, the clang flag. GCC has a similar flag named -mveclibabi.

Using -fveclib allowed me to compile Chapel code against SVML (--ccflags -fveclib=SVML) and gain about a 1.3x performance boost. With SVML, I additionally had to add -L/path/to/SVML/lib -lsvml to avoid linking issues. I could also link against other available vector libraries like libmvec for similar performance (--ccflags -fveclib=libmvec). This did not require extra linking args (because clang/gcc already know how to find libmvec).

However, all of this requires the C backend. When using the LLVM backend, fveclib is not respected.

I think we should add a Chapeltastic flag, like --vector-library=... to achieve this. With the C backend using clang, this would be equivalent to --ccflags -fveclib. With the C backend using gcc, -mveclibabi. With LLVM, we just need to thread in the right options. There is actually already a TODO in clangUtil.cpp about this.

There a few design questions here

1. What should the new flag name be?
2. What should the behavior be if the backend compiler doesn't support -fveclib or the requested library
3. Should we do some validation of the requested library, or just pass whatever we get along to the backend.
4. Should we try and come up with portable names across compilers?
   • e.g., in GCC the name is svml and in clang the name is SVML, with the wrong casing being rejected by the backend.

[mppf]

From https://llvm.org/docs/Vectorizers.html , it looks like other than SVML it might be any of these: accelerate,libmvec,massv,svml,sleef,darwin_libsystem_m,armpl,amdlibm

What should the new flag name be?

IMO --vector-library is a fine name for an option.

What should the behavior be if the backend compiler doesn't support -fveclib or the requested library

IMO, fail to compile

Should we do some validation of the requested library, or just pass whatever we get along to the backend.

IMO, pass whatever to the backend

Should we try and come up with portable names across compilers?

IMO we could do some point-adjustments here, e.g. fix the casing for SVML. I don't think we should make names in any way that assumes that we know the names of all of the vector libraries (i.e. I think it should be possible to use one that the backend C compiler / LLVM version supports but that we haven't heard of).

[jabraham17]

Thanks to @e-kayrakli, this does work today with the LLVM backend via --mllvm -vector-library=...

[mppf]

@jabraham17 -- I think it might be interesting to activate one of these vector libraries in performance playground testing so we can see if it's important enough to enough benchmarks/applications/code patterns that we should think about enabling it by default. I think we should be able to enable it by default with the default configuration using the LLVM backend because we should be able to bundle the source code for one of these that works with LLVM. (Of course there might be issues). But the first question is, how common is it to get a performance benefit from these?

[jabraham17]

I think it should be easy enough to default to libmvec without bundling anything, as I think most installs of glibc come with it.

If we are looking to bundle something, my goto pick would be SLEEF, but for whatever reason LLVM/clang doesn't auto vectorize to SLEEF on x86, its explicitly disallowed (even though SLEEF supports x86)

[damianmoz]

It is unfortunate that there is no easy and painless way of saying in pretty well any language that I want IEEE floating point behavior for most of my program but for the following functions in this part of the code I am happy to accept less that N ULP accuracy where N > 1, or I want 1.0 ULP accuracy (the norm), or I want correctly rounded (=0.5 ULP accuracy). Not 100% convinced a lot of those recommended operations in the IEEE 754 standard really should be there, rather than being in their own standard. Food for thought. The solution really needs addressing outside of Chapel as it is not a Chapel problem even if there are ways of tweaking the compiler and its flags to achieve a particular solution to part of the bigger problem.

There are bigger questions here about Data Parallel Types but I might leave it for now.