Checkpoint/Plotfile: Robust Bounds Calc

The calculation of the bounds in plotfiles (incl. checkpoints)
appears fishy. It relies on the order of the boxes in the
box array, which is not always pointing to a box on the
lower/upper bound of the simulation when load balancing is
used.

This can corrupt checkpoints (plotfiles) when writing, by
a) filtering out valid particles & fields and b) remembering
the wrong simulation meta-data for the simulation geometry
of fields/particles.

Co-authored-by: Andrew Myers <[email protected]>
Co-authored-by: Remi Lehe <[email protected]>
Co-authored-by: Davide Terzani <[email protected]>

Author: 4 people

Source/Diagnostics/FullDiagnostics.cpp

@@ -764,13 +764,16 @@ FullDiagnostics::InitializeBufferData (int i_buffer, int lev, bool restart ) {
         // Coarsen and refine so that the new BoxArray is coarsenable.
         ba.coarsen(m_crse_ratio).refine(m_crse_ratio);
 
+        // Box covering the extent of the user-defined diagnostic domain
+        amrex::Box const domain = ba.minimalBox();
+
         // Update the physical co-ordinates m_lo and m_hi using the final index values
         // from the coarsenable, cell-centered BoxArray, ba.
         for ( int idim = 0; idim < AMREX_SPACEDIM; ++idim) {
             diag_dom.setLo( idim, warpx.Geom(lev).ProbLo(idim) +
-                ba.getCellCenteredBox(0).smallEnd(idim) * warpx.Geom(lev).CellSize(idim));
+                domain.smallEnd(idim) * warpx.Geom(lev).CellSize(idim));
             diag_dom.setHi( idim, warpx.Geom(lev).ProbLo(idim) +
-                (ba.getCellCenteredBox( static_cast<int>(ba.size())-1 ).bigEnd(idim) + 1) * warpx.Geom(lev).CellSize(idim));
+                (domain.bigEnd(idim) + 1) * warpx.Geom(lev).CellSize(idim));
         }
 
     }