WaterLily on DelftBlue with 2025 software stack #2
Description
To install WaterLily on DElftBlue with the 2025 software stack and the navige Julia install, first load the 2025 software stack (this should be added to your ~/.bash_rcfile), and then loadJuliaandCUDA`.
[bob@login ~]$ module load julia cudayou can then check the Julia and CUDA versions with (this is important for later)
[bob@login ~]$ module list
Currently Loaded Modules:
1) slurm/current 2) DefaultModules 3) 2025 4) julia/1.11.6 5) cuda/11.6Once you have Julia and CUDA loaded you can start Julia`
[bob@login ~]$ julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.11.6 (2025-07-09)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> We can the enter the package manager by typing ] and the add CUDA, for compute capability issues, we need to used CUDA v5.8 and not the default.
(@v1.11) pkg> add CUDA@5.8
Installing known registries into `~/.julia`
Added `General` registry to ~/.julia/registries
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
Installed GPUArraysCore ─────────────── v0.2.0
Installed demumble_jll ──────────────── v1.3.0+0
Installed Crayons ───────────────────── v4.1.1
Installed Adapt ─────────────────────── v4.4.0
Installed ColorTypes ────────────────── v0.12.1
Installed Preferences ───────────────── v1.5.0
Installed Scratch ───────────────────── v1.3.0
Installed PrettyTables ──────────────── v2.4.0
Installed CUDA_Driver_jll ───────────── v13.0.1+0
Installed LLVMLoopInfo ──────────────── v1.0.0
Installed DataAPI ───────────────────── v1.16.0
Installed SentinelArrays ────────────── v1.4.8
Installed HashArrayMappedTries ──────── v0.2.0
Installed TableTraits ───────────────── v1.0.1
Installed Tables ────────────────────── v1.12.1
Installed GPUCompiler ───────────────── v1.7.2
Installed FixedPointNumbers ─────────── v0.8.5
Installed JLLWrappers ───────────────── v1.7.1
Installed PooledArrays ──────────────── v1.4.3
Installed InlineStrings ─────────────── v1.4.5
Installed AbstractFFTs ──────────────── v1.5.0
Installed Tracy ─────────────────────── v0.1.6
Installed StaticArrays ──────────────── v1.9.15
Installed PrecompileTools ───────────── v1.2.1
Installed IteratorInterfaceExtensions ─ v1.0.0
Installed StaticArraysCore ──────────── v1.4.3
Installed CUDA_Compiler_jll ─────────── v0.2.1+0
Installed CUDA_Runtime_Discovery ────── v1.0.0
Installed DataValueInterfaces ───────── v1.0.0
Installed LLVMExtra_jll ─────────────── v0.0.38+0
Installed NVTX ──────────────────────── v1.0.1
Installed OrderedCollections ────────── v1.8.1
Installed NVTX_jll ──────────────────── v3.2.2+0
Installed LaTeXStrings ──────────────── v1.4.0
Installed CEnum ─────────────────────── v0.5.0
Installed InvertedIndices ───────────── v1.3.1
Installed ScopedValues ──────────────── v1.5.0
Installed BFloat16s ─────────────────── v0.5.1
Installed Random123 ─────────────────── v1.7.1
Installed GPUToolbox ────────────────── v0.3.0
Installed GPUArrays ─────────────────── v11.2.6
Installed DataFrames ────────────────── v1.8.0
Installed JuliaNVTXCallbacks_jll ────── v0.2.1+0
Installed LibTracyClient_jll ────────── v0.9.1+6
Installed CUDA_Runtime_jll ──────────── v0.19.1+0
Installed Reexport ──────────────────── v1.2.2
Installed Statistics ────────────────── v1.11.1
Installed DataStructures ────────────── v0.19.1
Installed Colors ────────────────────── v0.13.1
Installed RandomNumbers ─────────────── v1.6.0
Installed ExprTools ─────────────────── v0.1.10
Installed Requires ──────────────────── v1.3.1
Installed MacroTools ────────────────── v0.5.16
Installed Compat ────────────────────── v4.18.1
Installed Missings ──────────────────── v1.2.0
Installed UnsafeAtomics ─────────────── v0.3.0
Installed StringManipulation ────────── v0.4.1
Installed SortingAlgorithms ─────────── v1.2.2
Installed KernelAbstractions ────────── v0.9.38
Installed LLVM ──────────────────────── v9.4.4
Installed CUDA ──────────────────────── v5.8.5
Installed Atomix ────────────────────── v1.1.2
Downloaded artifact: demumble
Downloaded artifact: CUDA_Driver
Downloaded artifact: LLVMExtra
Downloaded artifact: NVTX
Downloaded artifact: JuliaNVTXCallbacks
Downloaded artifact: LibTracyClient
Updating `~/.julia/environments/v1.11/Project.toml`
⌃ [052768ef] + CUDA v5.8.5
Updating `~/.julia/environments/v1.11/Manifest.toml`
Precompiling CUDA_Driver_jll...
3 dependencies successfully precompiled in 5 seconds. 23 already precompiled.
[621f4979] + AbstractFFTs v1.5.0
[79e6a3ab] + Adapt v4.4.0
[a9b6321e] + Atomix v1.1.2
⌅ [ab4f0b2a] + BFloat16s v0.5.1
[fa961155] + CEnum v0.5.0
⌃ [052768ef] + CUDA v5.8.5
[1af6417a] + CUDA_Runtime_Discovery v1.0.0
[3da002f7] + ColorTypes v0.12.1
[5ae59095] + Colors v0.13.1
[34da2185] + Compat v4.18.1
[a8cc5b0e] + Crayons v4.1.1
[9a962f9c] + DataAPI v1.16.0
[a93c6f00] + DataFrames v1.8.0
[864edb3b] + DataStructures v0.19.1
[e2d170a0] + DataValueInterfaces v1.0.0
[e2ba6199] + ExprTools v0.1.10
[53c48c17] + FixedPointNumbers v0.8.5
[0c68f7d7] + GPUArrays v11.2.6
[46192b85] + GPUArraysCore v0.2.0
[61eb1bfa] + GPUCompiler v1.7.2
⌅ [096a3bc2] + GPUToolbox v0.3.0
[076d061b] + HashArrayMappedTries v0.2.0
[842dd82b] + InlineStrings v1.4.5
[41ab1584] + InvertedIndices v1.3.1
[82899510] + IteratorInterfaceExtensions v1.0.0
[692b3bcd] + JLLWrappers v1.7.1
[63c18a36] + KernelAbstractions v0.9.38
[929cbde3] + LLVM v9.4.4
[8b046642] + LLVMLoopInfo v1.0.0
[b964fa9f] + LaTeXStrings v1.4.0
[1914dd2f] + MacroTools v0.5.16
[e1d29d7a] + Missings v1.2.0
[5da4648a] + NVTX v1.0.1
[bac558e1] + OrderedCollections v1.8.1
[2dfb63ee] + PooledArrays v1.4.3
⌅ [aea7be01] + PrecompileTools v1.2.1
[21216c6a] + Preferences v1.5.0
⌅ [08abe8d2] + PrettyTables v2.4.0
[74087812] + Random123 v1.7.1
[e6cf234a] + RandomNumbers v1.6.0
[189a3867] + Reexport v1.2.2
[ae029012] + Requires v1.3.1
[7e506255] + ScopedValues v1.5.0
[6c6a2e73] + Scratch v1.3.0
[91c51154] + SentinelArrays v1.4.8
[a2af1166] + SortingAlgorithms v1.2.2
[90137ffa] + StaticArrays v1.9.15
[1e83bf80] + StaticArraysCore v1.4.3
[10745b16] + Statistics v1.11.1
[892a3eda] + StringManipulation v0.4.1
[3783bdb8] + TableTraits v1.0.1
[bd369af6] + Tables v1.12.1
[e689c965] + Tracy v0.1.6
[013be700] + UnsafeAtomics v0.3.0
[d1e2174e] + CUDA_Compiler_jll v0.2.1+0
[4ee394cb] + CUDA_Driver_jll v13.0.1+0
[76a88914] + CUDA_Runtime_jll v0.19.1+0
[9c1d0b0a] + JuliaNVTXCallbacks_jll v0.2.1+0
[dad2f222] + LLVMExtra_jll v0.0.38+0
[ad6e5548] + LibTracyClient_jll v0.9.1+6
[e98f9f5b] + NVTX_jll v3.2.2+0
[1e29f10c] + demumble_jll v1.3.0+0
[0dad84c5] + ArgTools v1.1.2
[56f22d72] + Artifacts v1.11.0
[2a0f44e3] + Base64 v1.11.0
[ade2ca70] + Dates v1.11.0
[f43a241f] + Downloads v1.6.0
[7b1f6079] + FileWatching v1.11.0
[9fa8497b] + Future v1.11.0
[b77e0a4c] + InteractiveUtils v1.11.0
[4af54fe1] + LazyArtifacts v1.11.0
[b27032c2] + LibCURL v0.6.4
[76f85450] + LibGit2 v1.11.0
[8f399da3] + Libdl v1.11.0
[37e2e46d] + LinearAlgebra v1.11.0
[56ddb016] + Logging v1.11.0
[d6f4376e] + Markdown v1.11.0
[ca575930] + NetworkOptions v1.2.0
[44cfe95a] + Pkg v1.11.0
[de0858da] + Printf v1.11.0
[9a3f8284] + Random v1.11.0
[ea8e919c] + SHA v0.7.0
[9e88b42a] + Serialization v1.11.0
[2f01184e] + SparseArrays v1.11.0
[fa267f1f] + TOML v1.0.3
[a4e569a6] + Tar v1.10.0
[cf7118a7] + UUIDs v1.11.0
[4ec0a83e] + Unicode v1.11.0
[e66e0078] + CompilerSupportLibraries_jll v1.1.1+0
[deac9b47] + LibCURL_jll v8.6.0+0
[e37daf67] + LibGit2_jll v1.7.2+0
[29816b5a] + LibSSH2_jll v1.11.0+1
[c8ffd9c3] + MbedTLS_jll v2.28.6+0
[14a3606d] + MozillaCACerts_jll v2023.12.12
[4536629a] + OpenBLAS_jll v0.3.27+1
[bea87d4a] + SuiteSparse_jll v7.7.0+0
[83775a58] + Zlib_jll v1.2.13+1
[8e850b90] + libblastrampoline_jll v5.11.0+0
[8e850ede] + nghttp2_jll v1.59.0+0
[3f19e933] + p7zip_jll v17.4.0+2
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated -m`
Precompiling project...
72 dependencies successfully precompiled in 201 seconds. 30 already precompiled.Once you have successfuly installed CUDA, we can use it
julia> using CUDATo ensure that we use the CUDA from the system, we then need to specify the runtime version of CUDA, where the runtime version must be the same as the CUDA version you see in the loaded module list
julia> CUDA.set_runtime_version!(v"11.6")
[ Info: Configure the active project to use CUDA 11.6; please re-start Julia for this to take effect.We can then restart Julia and load CUDA again.
julia> using CUDA
Precompiling CUDA...
Info Given CUDA was explicitly requested, output will be shown live
Downloading artifact: CUDA_Compiler
4 dependencies successfully precompiled in 124 seconds. 97 already precompiled.
2 dependencies had output during precompilation:
┌ CUDA_Compiler_jll
│ Downloading artifact: CUDA_Runtime
└
┌ CUDA
│ [Output was shown above]
└ We are done with the CPU part of the installation. Now we must compile the GPU version of the code. The simplest for this is to request an interactive session on the GPU with
[bob@gpu001 ~]$ srun --mpi=pmix --job-name="int_gpu_job" --partition=gpu --time=00:35:00 --ntasks=1 --cpus-per-task=4 --gpus-per-task=1 --mem-per-cpu=4G --account=research-me-YOUR_GROUP --pty /bin/bash -ils
srun: job 8439927 queued and waiting for resources
srun: job 8439927 has been allocated resources
[bob@gpu001 ~]$ module load juliaWe can then make sure Julia and CUDA are loaded on the GPU (notice that now we are on @gpu0001 and not @login anymore
[bob@gpu001 ~]$ module load julia cuda
[bob@gpu001 ~]$ module list
Currently Loaded Modules:
1) slurm/current 2) DefaultModules 3) 2025 4) julia/1.11.6 5) cuda/11.6Simply launch Julia and import (using) CUDA
julia> using CUDA
using CUDA
Precompiling CUDA...
72 dependencies successfully precompiled in 158 seconds. 29 already precompiled.
1 dependency had output during precompilation:
┌ Atomix → AtomixCUDAExt
│ ┌ Warning: CUDA runtime library `libcublasLt.so.11` was loaded from a system path, `/beegfs/apps/generic/cuda-11.6/lib64/libcublasLt.so.11`.
│ │ This may cause errors.
│ │
│ │ If you're running under a profiler, this situation is expected. Otherwise,
│ │ ensure that your library path environment variable (e.g., `PATH` on Windows
│ │ or `LD_LIBRARY_PATH` on Linux) does not include CUDA library paths.
│ │
│ │ In any other case, please file an issue.
│ └ @ CUDA ~/.julia/packages/CUDA/TPbi4/src/initialization.jl:218
└
┌ Warning: CUDA runtime library `libcublasLt.so.11` was loaded from a system path, `/beegfs/apps/generic/cuda-11.6/lib64/libcublasLt.so.11`.
│ This may cause errors.
│
│ If you're running under a profiler, this situation is expected. Otherwise,
│ ensure that your library path environment variable (e.g., `PATH` on Windows
│ or `LD_LIBRARY_PATH` on Linux) does not include CUDA library paths.
│
│ In any other case, please file an issue.
└ @ CUDA ~/.julia/packages/CUDA/TPbi4/src/initialization.jl:218You can then check that we are using the binaries with
julia> CUDA.versioninfo()
CUDA toolchain:
- runtime 11.6, artifact installation
- driver 580.95.5 for 13.0
- compiler 11.8
CUDA libraries:
- CUBLAS: 11.9.2
- CURAND: 10.2.9
- CUFFT: 10.7.2
- CUSOLVER: 11.3.4
- CUSPARSE: 11.7.2
- CUPTI: 2022.1.1 (API 11.5.0)
- NVML: 13.0.0+580.95.5
Julia packages:
- CUDA: 5.8.5
- CUDA_Driver_jll: 13.0.1+0
- CUDA_Compiler_jll: 0.2.1+0
- CUDA_Runtime_jll: 0.19.1+0
Toolchain:
- Julia: 1.11.6
- LLVM: 16.0.6
Preferences:
- CUDA_Runtime_jll.version: 11.6
1 device:
0: Tesla V100S-PCIE-32GB (sm_70, 31.729 GiB / 32.000 GiB available)And you should be done. You can then add WaterLily and run your simulations.