-
Notifications
You must be signed in to change notification settings - Fork 662
Add GPU support to hypre solvers #14130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@victorapm Thank you! Let me have a chance to test these cases on our cluster and then we can merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you Victor for this PR! In this file and line
&PRES CHECK_POISSON=T, SOLVER='UGLMAT HYPRE', HYPRE_DEVICE_RUN=T, MAX_PRESSURE_ITERATIONS=1 /
Do we really need to set HYPRE_DEVICE_RUN? In cons.f90 it is already set to true, and if there is no DEVICE it will default to the CPU solver. I'm thinking we might not need to define HYPRE_DEVICE_RUN and have this as an input parameter, unless you can think of a reason to have it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Marcos, I agree it's unnecessary setting the variable in this case. The reason I put it there is to give an example of where this variable should be set in case someone builds hypre with GPU support but still wants to run it on CPUs. Then, in this case, HYPRE_DEVICE_RUN should be set to false. But, feel free to change any logic here as this is your code :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks victor, we'll test it here. The other question is, what HYPRE version do we need to work with the new routines added here (i.e. HYPRE_IJVECTORINITIALIZE_V2)?
marcosvanella
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A question that came up in terms of the coarsening/relaxation schemes defined in imkl.f90:
HYPRE_PRECOND_COARSENINGTYPE
HYPRE_PRECOND_SETRELAXTYPE
For CPUs we had 10 and 8 and I see in your PR you changed them to 8 and 18. Should we have different values if HYPRE is being run in GPU or CPU?
|
@victorapm This is not compiling with hypre 2.32.0, which looks to me like the current release. I get an error in Let me suggest that if we need a development branch of hypre that this should live in Marcos's FireX branch. |
| # Currently we need to refer directly to a commit which contains a | ||
| # patch to handle HYPRE_FMANGLE | ||
| GIT_TAG 6d01417697aa84d118eb1e1eb91315df42575be5 # v${HYPRE_GIT_VERSION} | ||
| GIT_TAG 63331f19c72b2ec363824907c0575a6edf7bbf3c # From: 01/27/25 # v${HYPRE_GIT_VERSION} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@marcosvanella @rmcdermo this is the minimum required version of hypre for this to work. This commit hash is located at hypre's master branch and will be added in the upcoming release
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@victorapm OK, understood. Thanks! We can add to FDS master branch once there is a new hypre release.
@marcosvanella Yes, we could have different defaults for CPU and GPU builds. Even though the new default leads to very similar performance as the old one (in terms of execution time) |
Add support to hypre solvers running on GPUs (NVIDIA, AMD and Intel)
GPU support can be controlled via input file with the variable
HYPRE_DEVICE_RUN. In case hypre is not compiled with GPU support, the solvers will run on CPUFDS+hypre build via CMake is ready, however the Makefile build needs work