Solver Strategy Adjustment for Efficient MultiApp CH-Stokes Coupling in 3D Simulations

[bo-qian]

Check these boxes if you have followed the posting rules.

• Q&A Navier Stokes is the most appropriate category for my question
• I have consulted the posting Guidelines on the Discussions front page
• I have searched the Discussions forum and my question has not been asked before
• I have searched the MOOSE website and the documentation does not answer my question
• I have formatted my post following the posting guidelines (avoid screenshots if possible, triple back quotes before/after pasted text, etc)

Question

Background Information

The governing equations are the Cahn–Hilliard (CH) phase-field equation and the incompressible Stokes equations, solved in a one-way coupled manner. The MultiApp capability is used:

• The master application solves the CH equation.
• The sub-application solves the incompressible Stokes equations.
  Since the coupling is one-way, the phase-field variable transferred from the master to the sub-application at each time step is treated as a constant parameter. All kernels are custom implemented.

---

Current Status

In the 2D case, the simulations work well using the following solver configuration:

[Preconditioning]
  [cw_coupling]
    type = SMP
    full = true
  []
[]

[Executioner]
  type = Steady
  solve_type = NEWTON
  verbose = true

  petsc_options_iname = '-pc_type -ksp_gmres_restart -pc_factor_mat_solver_type'
  petsc_options_value = 'lu 2500 superlu_dist'
  nl_rel_tol = 1e-15
  nl_abs_tol = 1e-6
[]

This setup yields very fast and robust convergence for 2D simulations.

However, in 3D, where the mesh can contain tens of millions of elements, direct solvers like LU with MUMPS or SuperLU result in memory overflow and cannot complete execution.

To address this, I tested the following alternative solver settings (still on a 2D problem but with modified solver options):

[Preconditioning]
  [cw_coupling]
    type = SMP
    full = true
  []
[]

[Executioner]
  type = Steady
  solve_type = NEWTON
  verbose = true

  petsc_options_iname = '-pc_type -pc_asm_overlap -sub_pc_type -sub_pc_factor_levels'
  petsc_options_value = 'asm      15               ilu          4'
  line_search = 'none'
  nl_rel_tol = 1e-12
  nl_abs_tol = 1e-13
  nl_max_its = 6
  l_tol = 1e-6
  l_max_its = 500
[]

This configuration is adapted from the example:
modules/navier_stokes/test/tests/finite_element/ins/lid_driven/lid_driven.i

Although it converges, convergence is slow, and the simulation requires a long time to meet the specified error tolerance.

---

Problem Statement

Given that the current direct solvers (e.g., LU) are not feasible for large-scale 3D simulations, is there a more efficient solver configuration that:

• Avoids direct solvers,
• Retains robustness,
• Improves convergence speed,

so that it can be directly applied to upcoming 3D coupled CH-Stokes simulations?

Two attached output files provide single-core performance logs for the two solver strategies described above.
asm.txt
lu.txt

Additional information

• Mesh size and type: 2D structured mesh. The velocity variables are discretized using second-order Lagrange elements, while the pressure variable uses first-order Lagrange elements (i.e., a Taylor–Hood element pairing).
• Reynolds number: Not applicable (creeping flow regime).
• Discretization method: Continuous Galerkin finite element method (CG-FEM).
• Physical model: Incompressible Stokes flow without turbulence or porous media.
• Solver method: The Stokes flow is solved in a one-way coupled fashion with the phase-field CH equation via the MultiApp mechanism. The CH equation is the master application, and the phase-field variable is transferred to the Stokes sub-application as a constant external field at each time step.
• Base input file: The simulation is initialized using a CH solution that defines the initial phase-field distribution. In the Stokes solver, this phase-field variable contributes to the momentum equations as a surface tension term.

[GiudGiud]

Hello

for 3D solves our most efficient fluid flow solver is the linear finite volume discretization of the incompressible Navier Stokes equations. This uses the SIMPLE/PIMPLE algorithm instead of a Newton solve.

Then there has been some successes using a field split to precondition the pressure-velocity equations. This should also scale better than LU to large simulations.
See the inputs in modules/navier_stokes/test/tests/finite_element/ins/lid_driven

[bo-qian]

You mean modules/navier_stokes/test/tests/finite_volume/ins/lid_driven?

no the SIMPLE implementation is in the segregated/ subfolders. The field split that @lindsayad mentioned is in the folder I mentioned

If so, how should I handle the data transfer of the intermediate variables between the two applications? Are there any reference materials or examples available?

There are plenty of options for passing fields. See this tutorial https://mooseframework.inl.gov/getting_started/examples_and_tutorials/tutorial02_multiapps/presentation/index.html#/

if the meshes are exactly the same between the two apps you can use the MultiAppCopyTransfer

Also, for the surface tension term in the Stokes equations that depends on the phase-field variable, would I need to implement a custom ADKernel to incorporate this contribution?

yes I think so, I dont know of kernel for this currently

I originally used the following MultiApp transfer settings:

[MultiApps]
  [./Stokes]
    type = FullSolveMultiApp
    input_files = "viscosity_sintering_Stokes_2D.i"
    execute_on = 'INITIAL TIMESTEP_END'
    clone_parent_mesh = true
  [../]
[]

[Transfers]
  [./CHToStokes]
    type = MultiAppCopyTransfer
    to_multi_app = Stokes
    source_variable = c
    variable = c
  [../]
  [./StokesToCH]
    type = MultiAppCopyTransfer
    from_multi_app = Stokes
    source_variable = 'u v p' 
    variable = 'u v p'
  [../]
[]

Both sets of governing equations are solved on the same mesh. So, if I solve the Cahn–Hilliard equation using the finite element method in the parent application, and switch to solving the Stokes equations using the finite volume method (INSFV) in the child application, can I keep the current transfer settings unchanged? Would it be sufficient to simply replace the solver in the child input with INSFV?

[GiudGiud]

Would it be sufficient to simply replace the solver in the child input with INSFV?

no because the FE solver uses nodal variables for velocity while the FV solver uses constant monomials.
You would need switch the variable types in CH to use constant monomials to present velocity as well

[lindsayad]

@bo-qian you changed the Schur complement solver settings. There is no need for using least-squares commutator preconditioning for a Stokes problem. The pressure mass matrix is spectrally equivalent to the Schur complement. You can pick a preconditioner that handles a mass matrix well. I used AMG in that input. You might even try Jacobi

[bo-qian]

@bo-qian you changed the Schur complement solver settings. There is no need for using least-squares commutator preconditioning for a Stokes problem. The pressure mass matrix is spectrally equivalent to the Schur complement. You can pick a preconditioner that handles a mass matrix well. I used AMG in that input. You might even try Jacobi

Thank you very much. I'll try out Jacobi to see if it can achieve the desired results.

[bo-qian]

Would it be sufficient to simply replace the solver in the child input with INSFV?

no because the FE solver uses nodal variables for velocity while the FV solver uses constant monomials. You would need switch the variable types in CH to use constant monomials to present velocity as well

Okay, I think I have a pretty clear idea about the details of the related operations now. Thank you very much!