Skip to content

Conversation

@Sbozzolo
Copy link
Member

@Sbozzolo Sbozzolo commented Oct 8, 2025

The tagged commit for hypre is incompatible with CUDA 13, so that Palace does not compile with the latest version of the CUDA toolkit.

This commit updates a few packages to their latest versions and adds a new dependency on Umpire. Umpire is strongly recommended for hypre 3 with CUDA. I also switch to using CMake instead of autotools in hypre and not use the internal hypre device manager.

Similar to magma, umpire is automatically added to the build when buidling with CUDA or HIP.

Closes #498
Spack build in docs depends on spack/spack-packages#1855 (or on pushing the package.py to spack-packages)

@Sbozzolo Sbozzolo force-pushed the gbozzola/hypre3 branch 4 times, most recently from 3593bd4 to 037c69f Compare October 8, 2025 23:29
@Sbozzolo Sbozzolo marked this pull request as ready for review October 9, 2025 22:40
Copy link
Contributor

@cameronrutherford cameronrutherford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor feedback. Overall great! 🥳

@victorapm
Copy link

Hi all, sorry for chiming in! Just wanted to note that hypre is moving to CMake as its recommended build system starting from hypre 3. Currently, hypre's CMake supports all build options that autotools' configure does and a few more. Build times is also generally faster with hypre's CMake

@Sbozzolo Sbozzolo force-pushed the gbozzola/hypre3 branch 5 times, most recently from bc40541 to c038ef6 Compare October 14, 2025 23:26
The tagged commit for `hypre` is incompatible with CUDA 13, so that
Palace does not compile with the latest version of the CUDA toolkit.

This commit updates a few packages to their latest versions and adds a
new dependency on Umpire. Umpire is strongly recommended for hypre 3
with CUDA.

Similar to magma, umpire is automatically added to the build when
buidling with CUDA or HIP.
@Sbozzolo
Copy link
Member Author

Sbozzolo commented Oct 15, 2025

I did more extensive testing of this PR and found that my previous implementation broke shared builds. This is due to a transitive dependency (blt) using a deprecated way to link against the CUDA runtime. I found easiest to just cherry-pick hypre-space/hypre#1380, which implements a fix for this and has already been approved. If the PR is merged soon, we can just bump the git tag for hypre.

Some followup work that will be left is:

  • The BLT patch I added to umpire has been merged upstream, but BLT is included as submodule of umpire with a specific git hash, so that the patch is not in umpire upstream yet. When umpire upgrades their BLT, we can remove the patch
  • There seems to be a performance regression with saving to Paraview from GPU. I will open a more detailed issue on this later.
  • Test-libceed crashes when Palace is compiled with CUDA (even when it is run on CPU)
  • Palace uses HYPRE_Int hypre_CSRMatrixInitialize, but there seems to be a v2 for that function (https://github.com/hypre-space/hypre/blob/fca98afcd78a26b784dc24b0c3edb0e6140598af/src/seq_mv/seq_mv.h#L434)

Sbozzolo added a commit that referenced this pull request Oct 20, 2025
PR #519 is almost ready, but it uncovered a couple of possible problems.
For this reason, we might want to be cautious with it.

One of the key benefits of PR #519 is that it brings compatibility with
CUDA 13 and with CUDA 12.9 + GCC 14. This PR does not fix compatibility
with CUDA 13, but it does for CUDA 12.9 + GCC 14.
@Sbozzolo
Copy link
Member Author

Test-libceed crashes when Palace is compiled with CUDA (even when it is run on CPU)

I looked a little bit into this. I found that the crash occurs at

mfem::forall(m + 1, [=] MFEM_HOST_DEVICE(int i) { d_I[i] = d_I_old[i]; });

in fem/libceed/operator.cpp. The pointers seem valid at that point, so I am not entirely sure what's up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update hypre to 3.0.0+ to fix compatibility with CUDA 13

3 participants