I am getting a strange CMake configure error with our AMReX app when we turn on -DAMReX_ASCENT=ON -DAMReX_CONDUIT=ON -DAMReX_GPU_BACKEND=CUDA.
It appears to be failing inside the BLT CMake logic to turn on CUDA. The full reproducer with a containerized build is:
$ git clone --recursive git@github.com:quokka-astro/quokka.git
$ cd quokka
$ singularity exec docker://ghcr.io/quokka-astro/quokka-arm64-cuda-ascent bash -c "mkdir build_cuda && cd build_cuda && cmake .. -DAMReX_SPACEDIM=3 -DAMReX_GPU_BACKEND=CUDA -DAMReX_ASCENT=ON -DAMReX_CONDUIT=ON -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -DAMReX_CUDA_ARCH=80"
INFO: Using cached SIF image
-- The CXX compiler identification is GNU 13.3.0
-- The C compiler identification is GNU 13.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CUDA compiler identification is NVIDIA 12.8.93 with host compiler GNU 13.3.0
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Quokka git branch: BenWibking/add-ascent-container
-- Quokka git commit hash: 84ba206bb81949d678c0b928121bd0ea8bd2538c
-- AMReX git commit hash: d3540b92d574148722bebe315ca50dd027e11aa0
-- CMake version: 4.0.1
-- AMReX installation directory: /usr/local
-- Building AMReX with AMReX_SPACEDIM = 3
-- Configuring AMReX with the following options enabled:
-- AMReX_PRECISION = DOUBLE
-- AMReX_GPU_BACKEND = CUDA
-- AMReX_GPU_MAX_THREADS = 256
-- AMReX_GPU_RDC
-- AMReX_MPI
-- AMReX_LINEAR_SOLVERS
-- AMReX_PARTICLES
-- AMReX_PARTICLES_PRECISION = DOUBLE
-- AMReX_CONDUIT
-- AMReX_ASCENT
-- AMReX_INLINE_LIMIT = 43210
-- AMReX_TINY_PROFILE
-- Enabled CUDA options:
-- AMReX_CUDA_FASTMATH
-- AMReX_CUDA_MAXREGCOUNT = 255
-- AMReX_CUDA_WARN_CAPTURE_THIS
-- AMReX_CUDA_SHOW_LINENUMBERS
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found MPI_C: /usr/lib/aarch64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /usr/lib/aarch64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: C CXX
-- Found CUDAToolkit: /usr/local/cuda/targets/sbsa-linux/include (found version "12.8.93")
-- Creating BLT MPI targets...
-- FindMPI Enabled (ENABLE_FIND_MPI == ON)
-- Found MPI: TRUE (found version "3.1")
-- BLT MPI Compile Flags:
-- BLT MPI Include Paths: /usr/lib/aarch64-linux-gnu/openmpi/include;/usr/lib/aarch64-linux-gnu/openmpi/include/openmpi
-- BLT MPI Libraries: /usr/lib/aarch64-linux-gnu/openmpi/lib/libmpi.so;/usr/lib/aarch64-linux-gnu/openmpi/lib/libmpi_cxx.so
-- BLT MPI Link Flags: SHELL:
-- MPI Executable: /usr/bin/mpiexec
-- MPI Num Proc Flag: -n
-- MPI Command Append:
-- Creating BLT CUDA targets...
CMake Error at /usr/local/ascent-checkout/lib/cmake/ascent/thirdparty/BLTSetupCUDA.cmake:26 (message):
CUDA language enabled prior to setting CMAKE_CUDA_HOST_COMPILER. Please
set CMAKE_CUDA_HOST_COMPILER prior to ENABLE_LANGUAGE(CUDA) or PROJECT(..
LANGUAGES CUDA)
Call Stack (most recent call first):
/usr/local/ascent-checkout/lib/cmake/ascent/BLTSetupTargets.cmake:97 (include)
/usr/local/ascent-checkout/lib/cmake/ascent/AscentConfig.cmake:156 (include)
extern/amrex/Tools/CMake/AMReXThirdPartyLibraries.cmake:107 (find_package)
extern/amrex/Src/CMakeLists.txt:49 (include)
I am getting a strange CMake configure error with our AMReX app when we turn on
-DAMReX_ASCENT=ON -DAMReX_CONDUIT=ON -DAMReX_GPU_BACKEND=CUDA.It appears to be failing inside the BLT CMake logic to turn on CUDA. The full reproducer with a containerized build is: