ECP 13: Improvements to linear solver setup costs

[spdomin]

Activities:

1. Benchmark, at scale, solver setup costs over the history of the project (~once per year).
2. Improve RAP operation.
3. Integrate with Kokkos and Tpetra.

[jhux2]

Activities:

1. Benchmark, at scale, solver setup costs over the history of the project (~once per year).
2. Improve RAP operation.
3. Integrate with Kokkos and Tpetra.

@spdomin I plan on tackling item 2, specifically the MPI scalability.

[spdomin]

Sounds great.

After the kickoff meeting, the team decided on moving forward in the NaluCFD product. I plan on moving all of the ECP issues over later this week.

[sthomas61]

Observation that the MueLu AMG set-up is called twice per time step (for each nonlinear iteration). If the matrices (Laplacian) are not changing within a time step then only one set-up per time step should be sufficient and re-use the AMG hierarchy. And this would eliminate 50% of set-up costs.

currently costs are 1 sec per set-up call for 256^3 ABL, 96 cores, 50 steps. run time 570 secs
100 set-up calls reduced to 50 would reduce time by 50 sec, and run in 520 sec. originally 640 sec