Skip to content

Enable MultiApp transfers between MFEM and libMesh problems on CPU#32315

Merged
nmnobre merged 72 commits intoidaholab:nextfrom
alexanderianblair:alexanderianblair/mfem-to-libmesh-transfers
Apr 29, 2026
Merged

Enable MultiApp transfers between MFEM and libMesh problems on CPU#32315
nmnobre merged 72 commits intoidaholab:nextfrom
alexanderianblair:alexanderianblair/mfem-to-libmesh-transfers

Conversation

@alexanderianblair
Copy link
Copy Markdown
Collaborator

@alexanderianblair alexanderianblair commented Feb 6, 2026

PR to enable transfers between transfers between MFEM GridFunctions and libMesh solutions on CPU, leveraging newly added GSLib interpolators for interpolation of MFEM GridFunctions at arbitrary sets of points.

Adds three new transfer classes, for transfers between arbitrary meshes in 2D and 3D using shape function evaluations for:

  • transfers from libMesh to MFEM, for first and second order LAGRANGE variables and constant MONOMIALs
  • transfers from MFEM to libMesh, for first and second order scalar H1 conforming variables and constant L2 vars,
  • transfers from MFEM to MFEM, for both scalar and vector FE types defined on H1, ND, RT, and L2 conforming FE spaces.

The newly added MFEM to MFEM transfer class is based off the MFEM gslib/field-interp miniapp. Interpolation and projection of libMesh variables in libMesh to MFEM and MFEM to libMesh transfers respectively is based off of existing implementations in MultiAppGeneralFieldTransfer and MultiAppShapeEvaluationTransfer.

Closes #31982

@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch 2 times, most recently from 9d7807b to a2fc66d Compare February 18, 2026 14:24
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch 3 times, most recently from 5087a0c to 5ceba23 Compare February 24, 2026 09:37
@alexanderianblair alexanderianblair changed the title WIP: Enable MultiApp transfers between MFEM and libMesh problems on CPU Enable MultiApp transfers between MFEM and libMesh problems on CPU Feb 25, 2026
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from a256a01 to 791693c Compare February 25, 2026 16:25
@moosebuild
Copy link
Copy Markdown
Contributor

moosebuild commented Feb 25, 2026

Job Documentation, step Docs: sync website on 58a80e6 wanted to post the following:

View the site here

This comment will be updated on new commits.

@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from a33274f to da45066 Compare February 25, 2026 19:54
@moosebuild
Copy link
Copy Markdown
Contributor

moosebuild commented Feb 26, 2026

Job Coverage, step Generate coverage on 58a80e6 wanted to post the following:

Framework coverage

953c57 #32315 58a80e
Total Total +/- New
Rate 85.87% 85.89% +0.02% 100.00%
Hits 132288 132622 +334 429
Misses 21776 21791 +15 0

Diff coverage report

Full coverage report

Modules coverage

Coverage did not change

Full coverage reports

Reports

This comment will be updated on new commits.

@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch 3 times, most recently from bcdd203 to 8c12ff0 Compare March 3, 2026 22:51
@alexanderianblair alexanderianblair marked this pull request as ready for review March 3, 2026 23:11
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from 8c12ff0 to 3057f8e Compare March 4, 2026 00:21
@moosebuild
Copy link
Copy Markdown
Contributor

Job Coverage, step Verify coverage on 3057f8e wanted to post the following:

The following coverage requirement(s) failed:

  • Failed to generate richards coverage rate (required: 93.0%)

1 similar comment
@moosebuild
Copy link
Copy Markdown
Contributor

Job Coverage, step Verify coverage on 3057f8e wanted to post the following:

The following coverage requirement(s) failed:

  • Failed to generate richards coverage rate (required: 93.0%)

@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch 2 times, most recently from 9f5bdad to 1dbcc43 Compare March 9, 2026 13:16
@moosebuild
Copy link
Copy Markdown
Contributor

Job Disable HDF5 on 1dbcc43 : invalidated by @loganharbour

Comment thread framework/include/mfem/transfers/MFEMMultiAppTransfer.h Outdated
Comment thread framework/include/mfem/transfers/MFEMMultiAppTransfer.h
Comment thread framework/include/mfem/transfers/MFEMMultiAppTransfer.h Outdated
Comment thread framework/include/mfem/transfers/MFEMMultiAppTransfer.h Outdated
Comment thread framework/include/mfem/utils/MFEMNodalProjector.h
Comment thread framework/include/mfem/transfers/MultiApplibMeshToMFEMShapeEvaluationTransfer.h Outdated
Comment thread framework/include/mfem/transfers/MultiApplibMeshToMFEMShapeEvaluationTransfer.h Outdated
Comment thread framework/include/mfem/transfers/MultiAppMFEMCopyTransfer.h Outdated
Comment thread framework/include/mfem/utils/MFEMVectorUtils.h Outdated
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch 2 times, most recently from 08dd19f to 7f1c81f Compare March 12, 2026 17:45
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from 7f1c81f to e4c4583 Compare March 24, 2026 17:59
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from 1b3394d to c1f2cf7 Compare April 23, 2026 20:27
@moosebuild
Copy link
Copy Markdown
Contributor

Job Precheck, step Clang format on c1f2cf7 wanted to post the following:

Your code requires style changes.

A patch was auto generated and copied here
You can directly apply the patch by running, in the top level of your repository:

curl -s https://mooseframework.inl.gov/docs/PRs/32315/clang_format/style.patch | git apply -v

Alternatively, with your repository up to date and in the top level of your repository:

git clang-format 93d5d4fe012870b41ba034e856781d1e421f0bf9

@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from c1f2cf7 to d5fab83 Compare April 23, 2026 20:48
Copy link
Copy Markdown
Member

@lindsayad lindsayad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm giving my approval contingent on my one remaining conversation being resolved, tests passing, and Nuno's review being met 😄

@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from e82400e to d6febef Compare April 24, 2026 01:30
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch 2 times, most recently from 17ee24c to 4d11729 Compare April 24, 2026 12:32
@alexanderianblair alexanderianblair force-pushed the alexanderianblair/mfem-to-libmesh-transfers branch from 4d11729 to 25b8962 Compare April 24, 2026 14:32
Copy link
Copy Markdown
Member

@nmnobre nmnobre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question:

  • What's the reason for using a different idiom to get libMesh's system in the libMesh->MFEM vs MFEM->libMesh transfer implementations?

One commit: nmnobre@5c259a0

  • Fixes a time section's (now) outdated description and a few of comments
  • Swaps the switch stmts with loops (sorry I didn't suggest this last time)
  • Reorders/simplifies some loop/if statements to reuse work or for early exit
  • Replaces getProblemData().cmplx_gridfunctions.Get() idioms with getComplexGridFunction()
  • Replaces dynamic_cast<MFEMProblem *>() with static_cast<MFEMProblem &>() where we already know it's safe to do so
  • Removes trailing whitespace in the tests file

Copy link
Copy Markdown
Member

@nmnobre nmnobre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though noting I'd like an answer above for completeness, I'm happy with this now.
Thanks for this Alex, very nice work.

In the future, besides the many-to-many transfers, it'd be cool if we could hide the type of transfer from the user, e.g. if the user wants a shape evaluation transfer, perhaps we could determine programatically if it needs to be libMesh->MFEM or MFEM->libMesh or MFEM->MFEM or libMesh->libMesh? Just a thought.

Copy link
Copy Markdown
Member

@nmnobre nmnobre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arrrgh... This (inadvertently I'm sure) reverts the change that @k-collie did to square.e in #32494, so we need to put that back (and perhaps rename square_quad.e as square_quad9.e?

@alexanderianblair
Copy link
Copy Markdown
Collaborator Author

One question:

* What's the reason for using a different idiom to get libMesh's system in the libMesh->MFEM vs MFEM->libMesh transfer implementations?

This could be changed - the reasons are historic. The

  • System * to_sys = find_sys(es, var_name); approach is based off of the similar usage in MultiAppGeneralFieldTransfer, and
  • System & from_sys = from_var.sys().system(); usage is based off of MultiAppGeneralFieldShapeEvaluationTransfer.

Since to_var is a const reference, using the second approach to determine to_sys in the MFEM->libMesh route would return a const reference to the system, which is not what we want here, as we need to set the DoFs on the new system. I generally think the latter is more expressive/readable though.

@alexanderianblair
Copy link
Copy Markdown
Collaborator Author

Arrrgh... This (inadvertently I'm sure) reverts the change that @k-collie did to square.e in #32494, so we need to put that back (and perhaps rename square_quad.e as square_quad9.e?

This was indeed a mistake - I've fixed this and renamed square_quad.e to square_quad9.e as suggested.

@moosebuild
Copy link
Copy Markdown
Contributor

Job Test, step Results summary on 58a80e6 wanted to post the following:

Framework test summary

Compared against 953c577 in job civet.inl.gov/job/3779372.

Removed tests

Test Time (s) Memory (MB)
mfem/transfers/h1_mfem_parent_mfem_sub.ComplexToSubFromParent/run 0.65 133.52
mfem/transfers/h1_mfem_parent_mfem_sub.FromSubToParent/run 0.64 130.62
mfem/transfers/h1_mfem_parent_mfem_sub.ComplexFromSubToParent/run 0.64 61.83
mfem/transfers/h1_mfem_parent_mfem_sub.ToSubFromParent/run 0.63 55.79
mfem/transfers/h1_mfem_sub_mfem_sub.test/run 0.61 131.52
mfem/transfers/h1_mfem_parent_mfem_sub.ComplexToSubFromParent/verify 0.06 0.00
mfem/transfers/h1_mfem_sub_mfem_sub.test/verify 0.04 0.00
mfem/transfers/h1_mfem_parent_mfem_sub.ToSubFromParent/verify 0.04 0.00
mfem/transfers/h1_mfem_parent_mfem_sub.ComplexFromSubToParent/verify 0.01 0.00
mfem/transfers/h1_mfem_parent_mfem_sub.FromSubToParent/verify 0.00 0.00
mfem/transfers/h1_mfem_sub_mfem_sub.error SKIP 0.00

Added tests

Test Time (s) Memory (MB)
mfem/transfers/sibling_transfers.MFEM_to_MFEM_sibling_transfers/1_to_N_nodal 0.89 126.83
mfem/transfers/sibling_transfers.libMesh_to_MFEM_sibling_transfers/1_to_N_nodal_parallel 0.86 166.94
mfem/transfers/sibling_transfers.MFEM_to_libMesh_sibling_transfers/1_to_N_nodal 0.86 153.38
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1TetToHexSecondOrder 0.86 124.85
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/VectorTransfer/Tetra 0.85 103.49
mfem/transfers/sibling_transfers.MFEM_to_MFEM_sibling_transfers/1_to_N_nodal_parallel 0.84 152.45
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/VectorTransfer/TetToHex 0.83 124.24
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1HexSecondOrder 0.81 140.64
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1TetSecondOrder 0.80 147.63
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1TriSecondOrder 0.80 154.48
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1TetToHexSecondOrder 0.79 117.28
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1HexSecondOrder 0.78 132.24
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1QuadSecondOrder 0.77 126.23
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1Tet 0.75 136.88
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1TriToQuadSecondOrder 0.75 136.60
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1TetSecondOrder 0.75 138.14
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1QuadSecondOrder 0.75 141.12
mfem/transfers/sibling_transfers.MFEM_to_MFEM_sibling_transfers/nodal_mixed_mesh 0.74 113.59
mfem/transfers/sibling_transfers.libMesh_to_MFEM_sibling_transfers/nodal_elem 0.73 102.99
mfem/transfers/sibling_transfers.MFEM_to_libMesh_sibling_transfers/nodal_elem 0.72 90.49
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1TriSecondOrder 0.72 127.48
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1TetToHex 0.72 147.67
mfem/transfers/sibling_transfers.libMesh_to_MFEM_sibling_transfers/nodal_mixed_mesh 0.71 127.25
mfem/transfers/sibling_transfers.libMesh_to_MFEM_sibling_transfers/elem_elem 0.71 85.95
mfem/transfers/sibling_transfers.MFEM_to_MFEM_sibling_transfers/nodal_nodal 0.71 104.27
mfem/transfers/displaced.MFEMTransferFromlibMeshDisplaced 0.71 155.45
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1TetToHex 0.71 81.80
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/VectorTransfer/Hex 0.71 96.87
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1Tri 0.71 130.47
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1Hex 0.70 95.41
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1TriToQuad 0.70 79.55
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1Tet 0.70 119.90
mfem/transfers/l2_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/L2Tet 0.70 164.43
mfem/transfers/sibling_transfers.MFEM_to_libMesh_sibling_transfers/nodal_nodal 0.69 97.99
mfem/transfers/sibling_transfers.libMesh_to_MFEM_sibling_transfers/nodal_nodal 0.69 109.63
mfem/transfers/sibling_transfers.MFEM_to_libMesh_sibling_transfers/elem_elem 0.69 107.32
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1Quad 0.68 130.23
mfem/transfers/displaced.MFEMTransferTolibMeshDisplaced 0.68 92.60
mfem/transfers/h1_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/H1Hex 0.67 105.25
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/OutOfMesh/Hex 0.67 102.86
mfem/transfers/l2_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/L2TetToHex 0.67 60.77
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1TriToQuad 0.67 108.27
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1Tri 0.66 64.82
mfem/transfers/l2_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/L2Hex 0.66 130.52
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1Quad 0.65 124.74
mfem/transfers/l2_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/L2Tet 0.65 130.06
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/ComplexFromSubToParent/run 0.64 103.33
mfem/transfers/l2_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/L2TetToHex 0.64 98.66
mfem/transfers/l2_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/L2Tri 0.63 70.17
mfem/transfers/l2_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/L2TriToQuad 0.63 131.39
mfem/transfers/mfem_sub_mfem_sub.MFEMCopyTransfer/test/run 0.63 125.47
mfem/transfers/l2_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/L2TriToQuad 0.63 52.93
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/ToSubFromParent/run 0.63 67.40
mfem/transfers/l2_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/L2Quad 0.63 81.74
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/FromSubToParent/run 0.63 87.02
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/ComplexToSubFromParent/run 0.62 88.59
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/FromSubToParent/run 0.62 139.48
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/ComplexFromSubToParent/run 0.62 144.93
mfem/transfers/l2_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/L2Quad 0.62 97.14
mfem/transfers/l2_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/L2Tri 0.62 65.43
mfem/transfers/h1_mfem_parent_libmesh_sub.MFEMTransferFromlibMesh/H1TriToQuadSecondOrder 0.61 101.19
mfem/transfers/l2_libmesh_parent_mfem_sub.MFEMTransferTolibMesh/L2Hex 0.61 80.02
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/ComplexFromSubToParent/verify 0.13 0.00
mfem/transfers/mfem_sub_mfem_sub.MFEMCopyTransfer/test/verify 0.04 0.00
mfem/transfers/mfem_parent_mfem_sub.MFEMShapeEvaluationTransfer/FromSubToParent/verify 0.03 0.00
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/ComplexToSubFromParent/verify 0.02 0.00
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/ToSubFromParent/verify 0.02 0.00
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/FromSubToParent/verify 0.01 0.00
mfem/transfers/mfem_parent_mfem_sub.MFEMCopyTransfer/ComplexFromSubToParent/verify 0.01 0.00
mfem/transfers/mfem_sub_mfem_sub.MFEMCopyTransfer/error SKIP 0.00

Modules test summary

Compared against 953c577 in job civet.inl.gov/job/3779372.

No change

@nmnobre nmnobre merged commit de74ae5 into idaholab:next Apr 29, 2026
68 of 69 checks passed
nmnobre added a commit to nmnobre/moose that referenced this pull request Apr 30, 2026
nmnobre added a commit to nmnobre/moose that referenced this pull request Apr 30, 2026
@loganharbour
Copy link
Copy Markdown
Member

The sweeps should be added to anything relating to transfers, meshing, or parallel algorithms.

This broke next: https://civet.inl.gov/event/292493/

nmnobre added a commit to nmnobre/moose that referenced this pull request May 3, 2026
nmnobre added a commit to nmnobre/moose that referenced this pull request May 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libMesh/MFEM transfers

7 participants