Suggestion: Use llvm 15 instead of Apple clang to compile suite-sparse

[pcostanza]

Output of brew config

HOMEBREW_VERSION: 4.0.4
ORIGIN: https://github.com/Homebrew/brew
HEAD: 4d0154fa7926e8a7b06e9b074dd45957e820407c
Last commit: 4 days ago
Core tap origin: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 1d8c58a0aad7953501bd69df9f5014a6dfbf199e
Core tap last commit: 2 weeks ago
Core tap branch: master
Core tap JSON: 03 Mar 14:49 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :0
HOMEBREW_EDITOR: emacs
HOMEBREW_MAKE_JOBS: 12
HOMEBREW_NO_GOOGLE_ANALYTICS: set
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: dodeca-core 64-bit kabylake
Clang: 14.0.0 build 1400
Git: 2.37.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.86.0 => /usr/bin/curl
macOS: 13.2.1-x86_64
CLT: 14.2.0.0.1.1668646533
Xcode: 14.2

Output of brew doctor

"Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!"

-> I'm not worried. :) This just reports some unbrewed dylibs, header files, and static libraries, for example my handmade version of suite-sparse.

Description of issue

It seems that suite-sparse is currently compiled with Apple clang. That leads to a version of suite-sparse that doesn't have OpenMP enabled, so only runs in single-threaded mode. If you compile suite-sparse with llvm 15 (and libomp), as provided by Homebrew, then OpenMP is enabled and can use multiple threads for improved performance. (I verified this with the GraphBLAS library included in suite-sparse.) So my suggestion is to do that.

I hope this is the correct category for making such a suggestion. This is my first time posting here, so apologies if this is inappropriate.

Cheers,
Pascal

[carlocab]

Hi @pcostanza, thanks for the note. Is using the whole LLVM compiler suite required for this? i.e. can we just use the separately packaged libomp formula?

[pcostanza]

Hi, thanks for your response. When I tried with installing just libomp, I couldn't make it work. But I don't know if I did anything wrong. With the whole LLVM compiler suite, it worked.

[carlocab]

Could you describe in more detail the steps that you attempted, both with libomp and the full llvm?

[pcostanza]

The description for libomp says the following: 'libomp is keg-only, which means it was not symlinked into /usr/local,
because it can override GCC headers and result in broken builds.

For compilers to find libomp you may need to set:
export LDFLAGS="-L/usr/local/opt/libomp/lib"
export CPPFLAGS="-I/usr/local/opt/libomp/include"'

When I do that, and then try to make SuiteSparse:GraphBLAS, the output states that it doesn't find OpenMP. I haven't investigated this further from here.

With LLVM (now version 16.0.0), I have the following configuration:

export LLVM_PATH="/usr/local/opt/llvm/"
export LLVM_VERSION="16.0.0"
export PATH="$LLVM_PATH:$PATH"
export LD_LIBRARY_PATH="$LLVM_PATH/lib/:$LD_LIBRARY_PATH"
export DYLD_LIBRARY_PATH="$LLVM_PATH/lib/:$DYLD_LIBRARY_PATH"
export CPATH="$LLVM_PATH/lib/clang/$LLVM_VERSION/include/"
export LDFLAGS="-L$LLVM_PATH/lib"
export CPPFLAGS="-I$LLVM_PATH/include"
export CC="$LLVM_PATH/bin/clang"
export CXX="$LLVM_PATH/bin/clang++"

When I then issue make in the SuiteSparse/GraphBLAS folder, everything goes smoothly.

Here is a little test program to check whether SuiteSparse is using OpenMP:

#include "GraphBLAS.h"

int main(int argc, char **argv) {
GrB_init(GrB_NONBLOCKING);
bool omp;
GxB_get(GxB_LIBRARY_OPENMP, &omp);
printf("OpenMP: %s\n", omp ? "true" : "false");
GrB_finalize();
}

(Use -lgraphblas.)

I hope this helps.

Pascal

[carlocab]

Nice, thanks. Do you have any benchmarks for the speed-up from enabling OpenMP?

[pcostanza]

There are GraphBLAS benchmarks in https://github.com/GraphBLAS/LAGraph (see https://github.com/GraphBLAS/LAGraph/tree/stable/src/benchmark).

[carlocab]

I don't use suite-sparse, so I'm not so invested as to run the benchmarks myself, but if you have benchmarking results that show significant improvement from enabling OpenMP then I'm willing to look at building suite-sparse with it.

[pcostanza]

Ok, I understand, and I apologise for not having provided a more complete answer. There is a paper describing the performance improvements from OpenMP from 2020. See https://people.engr.tamu.edu/davis/publications_files/CSC20_OpenMP_GraphBLAS.pdf - Table 2 in that paper shows significant speedups, depending on algorithm, with up to 30x faster with 40 threads compared to one thread for Triangle Counting. That's why the user guide for SuiteSparse:GraphBLAS (https://github.com/DrTimothyAldenDavis/SuiteSparse/blob/dev/GraphBLAS/Doc/GraphBLAS_UserGuide.pdf) explicitly recommends compiling with OpenMP. See Section 8.1 in the user guide (page 168), where it states the following: "Compiling GraphBLAS without OpenMP is not recommended for installation in a package manager (Linux, conda-forge, spack, brew, vcpkg, etc)."

I hope this helps.

[pcostanza]

One way to install libomp on Mac seems to be the following:

        brew tap-new libomp/cask
        brew extract --version=14.0.6 libomp libomp/cask
        brew install [email protected]

I haven't tested this myself yet, though.

[SMillerDev]

If you need that specific version, sure. If you need any version you can just use the normal formula.

[cho-m]

Using standard libomp formula should work (e.g. Homebrew/homebrew-core#129100). In Homebrew, the main issue with enabling OpenMP is trying to resolve situation of which implementation (LLVM libomp vs GCC libgomp) and the potential risks of mixing.