Block restriction of libMesh unstructured mesh tallies #3694
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
libMesh meshes support the use of subdomains of elements known as "blocks". Often times in multi-physics applications (such as Cardinal) users will mesh their entire geometries, but only need to compute specific physics over a subset of the blocks in the mesh. Presently, OpenMC does not support the specification of block restriction for libMesh unstructured mesh tallies, and tallying on the entire geometry can be quite compute-intensive (especially for scores like kappa-fission which are defined over a single block). To get around this in Cardinal we've been creating a clone of the full libMesh mesh which only contains the elements belonging to the blocks the user wishes to tally on. This is a rather memory-intensive workaround, which becomes quite expensive when scaling up to large problems.
This PR implements a more elegant solution which uses the infrastructure in
AdaptiveLibMesh. The forward and inverse mapping between bins and elements is redefined to work for adaptive meshes and block restriction such that tallying is only performed on the requested subdomains.In addition to the above, this PR also changes how libMesh scaling works. The previous approach scaled all of the nodes in the libMesh mesh ahead of time. This works when OpenMC runs as a standalone application, however it causes problems when the mesh is managed by an external application which may assume that the mesh doesn't change after initialization. In the case of Cardinal, MOOSE assumes that the mesh is not modified after setup and scaling the mesh in-place causes failures with other MOOSE systems. An alternative has been implemented here which uses the inverse scaling transformation to scale down sample points into mesh-space, instead of using the transformation to scale up the mesh.
If preferred, I can separate these two Cardinal-motivated changes into different PR's. Just thought I'd lump them together since the scaling change is rather small.
cc'ing @aprilnovak
Checklist
I have followed the style guidelines for Python source files (if applicable)I have made corresponding changes to the documentation (if applicable)I have added tests that prove my fix is effective or that my feature works (if applicable)