Skip to content

Commit 9e7d0c9

Browse files
plavinplavin
authored
Merge pull request #253 from hpcgarage/spatter-fixes
Spatter Build Instructions and Intel Backend Fixes
2 parents db4a95a + 9df2e4d commit 9e7d0c9

File tree

3 files changed

+220
-72
lines changed

3 files changed

+220
-72
lines changed

‎Build.md‎

Lines changed: 144 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,144 @@
1-
# Supported configurations
2-
3-
## Serial
4-
### Supported Compilers
5-
* gnu
6-
* cray
7-
8-
# Cuda
9-
### Supported Compilers and optional arguments
10-
* nvcc
11-
* `-DCUDA_ARCH=<ARCH>`
12-
* Default is 7.0
13-
* For example to set it to 7.0, enter `-DCUDA_ARCH=70`
14-
15-
## Openmp
16-
### Supported Compilers and optional arguments
17-
* gnu
18-
* `-DUSE_MPI=1`
19-
* cray
20-
* `-DUSE_PAPI=1`
21-
* `-DUSE_SVE=1`
22-
* clang
23-
* armclang (wombat)
24-
* xl
25-
* intel
26-
* `-DINTEL_PLATFORM=<PLATFORM>`
27-
* skylake
28-
* avx_crossplatform
29-
* non_avx
30-
* `-DUSE_MPI=1`
31-
* `-DUSE_PAPI=1`
1+
# Building Spatter
2+
3+
Spatter uses CMake (currently ≥ 3.25) and requires a C++17-compatible compiler.
4+
5+
## Quick Start
6+
7+
```bash
8+
cmake -B build [options]
9+
cmake --build build -j
10+
```
11+
12+
The compiled `spatter` binary will be placed in the `build/` directory in this example.
13+
14+
## Installation
15+
16+
```bash
17+
cmake --install build --prefix /path/to/install
18+
```
19+
20+
This installs the `spatter` and `gz_read` binaries to `<prefix>/bin/`.
21+
22+
## Spack Installation
23+
You can also use Spack to install the default (OpenMP) and CUDA versions of Spatter. Please see the [Spack packages](https://packages.spack.io/package.html?name=spatter) for more information.
24+
25+
---
26+
27+
## CMake Options
28+
29+
| Option | Type | Default | Description |
30+
|---|---|---|---|
31+
| `USE_CUDA` | `BOOL` | `OFF` | Enable NVIDIA CUDA backend |
32+
| `USE_HIP` | `BOOL` | `OFF` | Enable AMD HIP/ROCm backend |
33+
| `USE_ONEAPI` | `BOOL` | `OFF` | Enable Intel OneAPI SYCL backend |
34+
| `USE_OPENMP` | `BOOL` | `OFF` | Enable OpenMP threading |
35+
| `USE_MPI` | `BOOL` | `OFF` | Enable MPI support |
36+
| `CMAKE_BUILD_TYPE` | `STRING` | `Release` | Build type: `Release`, `Debug`, `RelWithDebInfo` |
37+
| `CMAKE_CXX_COMPILER` | `STRING` | system default | C++ compiler to use |
38+
39+
> **Note:** `USE_CUDA`, `USE_HIP`, and `USE_ONEAPI` are mutually exclusive. Only one GPU backend may be enabled at a time.
40+
41+
## JSON Support
42+
JSON support via `nlohmann/json v3.11.2` is automatically fetched via CMake's `FetchContent`.
43+
44+
---
45+
46+
## Backend Build Examples
47+
48+
> **Note:** We recommend using the syntax `build_<backend>` to build Spatter backends, which makes it easier to distinguish different variants of the executable.
49+
50+
51+
### Serial (CPU only)
52+
53+
Any supported C++ compiler works, and no backend flags are required.
54+
55+
```bash
56+
cmake -B build
57+
cmake --build build -j
58+
```
59+
60+
### OpenMP
61+
62+
```bash
63+
cmake -B build_omp -DUSE_OPENMP=ON
64+
cmake --build build_omp -j
65+
```
66+
67+
The OpenMP backend can be combined with the MPI backend:
68+
69+
```bash
70+
cmake -B build_omp_mpi -DUSE_OPENMP=ON -DUSE_MPI=ON
71+
cmake --build build_omp_mpi -j
72+
```
73+
74+
### CUDA (NVIDIA GPUs)
75+
76+
The CUDA backend requires the CUDA Toolkit and a CUDA-capable compiler (`nvcc`) to be on `PATH`. CMake will auto-detect the installed CUDA toolkit.
77+
78+
```bash
79+
cmake -B build_cuda -DUSE_CUDA=ON
80+
cmake --build build_cuda -j
81+
```
82+
If multiple CUDA versions are installed, you can point CMake at the preferred one as follows:
83+
84+
```bash
85+
cmake -B build_cuda12 -DUSE_CUDA=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
86+
cmake --build build_cuda12 -j
87+
```
88+
89+
### HIP (AMD GPUs)
90+
91+
The HIP backend requires the ROCm Toolkit and a HIP-capable C++ compiler (`hipcc`) to be on `PATH`. The default ROCm path is `/opt/rocm`, but this can be overriden with `-DHIP_PATH=<path>`.
92+
93+
```bash
94+
cmake -B build_hip -DUSE_HIP=ON -DCMAKE_CXX_COMPILER=hipcc
95+
cmake --build build_hip -j
96+
```
97+
98+
With a non-default ROCm installation:
99+
100+
```bash
101+
cmake -B build_hip -DUSE_HIP=ON -DCMAKE_CXX_COMPILER=hipcc -DHIP_PATH=/opt/<rocm-custom>
102+
cmake --build build_hip -j
103+
```
104+
105+
### OneAPI (Intel GPUs / SYCL)
106+
107+
The OneAPI backend requires the OneAPI Toolkit and a SYCL-capable C++ compiler (`icpx`) to be on `PATH`. You can typically set `icpx` to be on your path using environment variables or the `setvars.sh` script included with OneAPI installations.
108+
109+
```bash
110+
cmake -B build_oneapi -DUSE_ONEAPI=ON -DCMAKE_CXX_COMPILER=icpx
111+
cmake --build build_oneapi -j
112+
```
113+
114+
---
115+
116+
## Supported Compilers
117+
118+
| Compiler | ID | Serial | OpenMP | CUDA | HIP | OneAPI |
119+
|---|---|:---:|:---:|:---:|:---:|:---:|
120+
| GCC | `GNU` | ✓ | ✓ | | | |
121+
| Clang | `Clang` | ✓ | ✓ | | | |
122+
| Intel (`icpx`) | `IntelLLVM` | ✓ | ✓ | | | ✓ |
123+
| Cray CCE | `Cray` | ✓ | ✓ | | | |
124+
| IBM XL | `XL` | ✓ | ✓ | | | |
125+
| NVCC | `NVIDIA` | | | ✓ | | |
126+
| `hipcc` | `ROCm (Clang)) | | | | ✓ | |
127+
128+
---
129+
130+
## Build Types
131+
132+
The default build type is `Release`. To change it:
133+
134+
```bash
135+
cmake -B build -DCMAKE_BUILD_TYPE=Debug
136+
```
137+
or
138+
```bash
139+
cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
140+
```
141+
142+
Debug builds include `-g -debug all`. GCC and Clang builds enable `-Wall -Wextra -Wconversion -pedantic-errors`.
143+
144+
---

‎CMakeLists.txt‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ project(Spatter VERSION 2.2.0 LANGUAGES CXX)
77

88
option(USE_CUDA "Enable CUDA support" OFF)
99
option(USE_HIP "Enable HIP support" OFF)
10-
option(USE_ONEAPI "Enable HIP support" OFF)
10+
option(USE_ONEAPI "Enable OneAPI support" OFF)
1111

1212
set(SELECTED_COUNT 0)
1313

@@ -48,8 +48,9 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)
4848
include(pkgs/JSONSupport)
4949
include(pkgs/MPISupport)
5050
include(pkgs/OpenMPSupport)
51-
include(pkgs/CUDASupport)
52-
include(pkgs/OneAPISupport)
51+
if(USE_ONEAPI)
52+
include(pkgs/OneAPISupport)
53+
endif()
5354
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -g -debug all")
5455

5556
if (APPLE)

‎cmake/pkgs/OneAPISupport.cmake‎

Lines changed: 72 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,88 @@
1-
# Option to enable OneAPI
2-
option(USE_ONEAPI "Enable support for Intel OneAPI" ON)
1+
# OneAPISupport.cmake
2+
#
3+
# IMPORTANT: The Intel DPC++/C++ compiler (icpx) must be selected BEFORE
4+
# CMake's project() call. Pass it on the command line:
5+
# cmake -DCMAKE_CXX_COMPILER=icpx ...
6+
# or source Intel's environment script first:
7+
# source /opt/intel/oneapi/setvars.sh && cmake -DCMAKE_CXX_COMPILER=icpx ...
38

49
if (USE_ONEAPI)
5-
# Check for compiler and print debug info
6-
include(CheckLanguage)
7-
check_language(CXX)
8-
9-
message(STATUS "CMAKE_CXX_COMPILER: ${CMAKE_CXX_COMPILER}")
10-
message(STATUS "CMAKE_CXX_COMPILER_ID: ${CMAKE_CXX_COMPILER_ID}")
11-
12-
# Explicitly check if the compiler is IntelLLVM (DPC++)
13-
if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "IntelLLVM")
14-
message(STATUS "OneAPI DPC++ compiler (IntelLLVM) detected: ${CMAKE_CXX_COMPILER}")
10+
# Verify that icpx (IntelLLVM) is actually the active CXX compiler.
11+
# CMAKE_CXX_COMPILER cannot be changed after project() has been called.
12+
if (NOT "${CMAKE_CXX_COMPILER_ID}" STREQUAL "IntelLLVM")
13+
message(FATAL_ERROR
14+
"USE_ONEAPI requires the Intel DPC++/C++ compiler (icpx/icx).\n"
15+
" Detected: CMAKE_CXX_COMPILER_ID=${CMAKE_CXX_COMPILER_ID}"
16+
" (${CMAKE_CXX_COMPILER})\n"
17+
" Reconfigure with: -DCMAKE_CXX_COMPILER=icpx\n"
18+
" or source Intel's setvars.sh before running cmake.")
19+
endif()
1520

16-
# Set compiler flags
17-
set(CMAKE_CXX_STANDARD 17)
18-
set(CMAKE_CXX_STANDARD_REQUIRED ON)
21+
message(STATUS "OneAPI DPC++ compiler (IntelLLVM) detected: ${CMAKE_CXX_COMPILER}")
1922

20-
# Define a preprocessor directive for OneAPI support
21-
add_definitions(-DUSE_ONEAPI)
23+
# -fsycl is required for SYCL device compilation
24+
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsycl")
2225

23-
# First try finding IntelSYCL (New recommended package)
24-
set(IntelSYCL_DIR "/net/projects/tools/x86_64/rhel-8/intel-oneapi/2024.2/compiler/latest/lib/cmake/sycl")
25-
find_package(IntelSYCL REQUIRED)
26+
add_definitions(-DUSE_ONEAPI)
2627

27-
if (IntelSYCL_FOUND)
28-
message(STATUS "Intel OneAPI SYCL package found.")
29-
set(COMMON_LINK_LIBRARIES ${COMMON_LINK_LIBRARIES} IntelSYCL::SYCL_CXX)
28+
# ------------------------------------------------------------------
29+
# Locate the IntelSYCL CMake package.
30+
# Search order:
31+
# 1. User-supplied IntelSYCL_DIR cache variable
32+
# 2. CMPLR_ROOT environment variable (set by setvars.sh / modulefiles)
33+
# 3. ONEAPI_ROOT environment variable (set by setvars.sh)
34+
# 4. Derive from the compiler executable location
35+
# ------------------------------------------------------------------
36+
if (NOT DEFINED IntelSYCL_DIR)
37+
if (DEFINED ENV{CMPLR_ROOT})
38+
set(IntelSYCL_DIR "$ENV{CMPLR_ROOT}/lib/cmake/sycl"
39+
CACHE PATH "Path to IntelSYCL CMake config")
40+
elseif (DEFINED ENV{ONEAPI_ROOT})
41+
set(IntelSYCL_DIR "$ENV{ONEAPI_ROOT}/compiler/latest/lib/cmake/sycl"
42+
CACHE PATH "Path to IntelSYCL CMake config")
3043
else()
31-
message(WARNING "IntelSYCL::SYCL target not found! Falling back to manual linking.")
32-
33-
# Manually link the Intel SYCL library
34-
set(SYCL_LIB_PATH "/net/projects/tools/x86_64/rhel-8/intel-oneapi/2024.2/compiler/2024.2/lib")
44+
# Fall back to a path relative to the compiler binary
45+
get_filename_component(_icpx_bindir "${CMAKE_CXX_COMPILER}" DIRECTORY)
46+
set(IntelSYCL_DIR "${_icpx_bindir}/../lib/cmake/sycl"
47+
CACHE PATH "Path to IntelSYCL CMake config")
48+
endif()
49+
endif()
3550

36-
# Correct OneAPI include path
37-
set(SYCL_INCLUDE_PATH "/net/projects/tools/x86_64/rhel-8/intel-oneapi/2024.2/compiler/2024.2/include")
51+
find_package(IntelSYCL QUIET)
3852

39-
include_directories(${SYCL_INCLUDE_PATH})
40-
link_directories(${SYCL_LIB_PATH})
53+
if (IntelSYCL_FOUND)
54+
message(STATUS "Intel OneAPI SYCL package found (${IntelSYCL_DIR})")
55+
set(COMMON_LINK_LIBRARIES ${COMMON_LINK_LIBRARIES} IntelSYCL::SYCL_CXX)
56+
else()
57+
message(WARNING
58+
"IntelSYCL CMake package not found (searched: ${IntelSYCL_DIR}).\n"
59+
"Falling back to manual SYCL library linking.")
4160

42-
set(COMMON_LINK_LIBRARIES ${COMMON_LINK_LIBRARIES} "${SYCL_LIB_PATH}/libsycl.so")
61+
# Derive SYCL lib/include paths with the same priority order
62+
if (DEFINED ENV{CMPLR_ROOT})
63+
set(_sycl_root "$ENV{CMPLR_ROOT}")
64+
elseif (DEFINED ENV{ONEAPI_ROOT})
65+
set(_sycl_root "$ENV{ONEAPI_ROOT}/compiler/latest")
66+
else()
67+
get_filename_component(_icpx_bindir "${CMAKE_CXX_COMPILER}" DIRECTORY)
68+
set(_sycl_root "${_icpx_bindir}/..")
4369
endif()
4470

45-
# Force CMake to use DPC++ compiler
46-
set(CMAKE_CXX_COMPILER icpx)
47-
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsycl")
71+
set(SYCL_LIB_PATH "${_sycl_root}/lib")
72+
set(SYCL_INCLUDE_PATH "${_sycl_root}/include")
4873

49-
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I${SYCL_INCLUDE_PATH}")
74+
if (NOT EXISTS "${SYCL_LIB_PATH}/libsycl.so")
75+
message(FATAL_ERROR
76+
"Could not find libsycl.so under ${SYCL_LIB_PATH}.\n"
77+
"Set CMPLR_ROOT or ONEAPI_ROOT, or specify -DIntelSYCL_DIR=<path>.")
78+
endif()
5079

51-
else()
52-
message(FATAL_ERROR "OneAPI DPC++ compiler not found. Detected CMAKE_CXX_COMPILER_ID=${CMAKE_CXX_COMPILER_ID}")
80+
include_directories(${SYCL_INCLUDE_PATH})
81+
link_directories(${SYCL_LIB_PATH})
82+
set(COMMON_LINK_LIBRARIES ${COMMON_LINK_LIBRARIES} "${SYCL_LIB_PATH}/libsycl.so")
83+
message(STATUS "Using manual SYCL library: ${SYCL_LIB_PATH}/libsycl.so")
84+
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I${SYCL_INCLUDE_PATH}")
5385
endif()
86+
87+
message(STATUS "OneAPI support enabled (flags: ${CMAKE_CXX_FLAGS})")
5488
endif()

0 commit comments

Comments
 (0)