Gauge Values for Aux Arrays Incorrect

[mandli]

As discussed in #341 and #649, gauge values near patch boundaries are periodically incorrect. This is fixed by changing the time when update_gauges is called, but as outlined by @rjleveque in #649 this is problematic for a number of reasons. Instead we can add time to setaux calls, but this does not completely solve the problem as somewhere in the algorithm to fill in ghost cells, this is filling in with incorrect data. This issue bring up a few points:

• Do we want to put time into setaux? This was not nearly as hard as I thought but I was not able to guarantee that whatever time is there is correct.
• There are a number of places where we copy aux values from the old grids (mostly for topography). This was modified to handle moving topography, but not general time-dependent aux values. While I do not think the problem here is directly impacted by this, we may need to reconsider this point.
• This may impact how @kbarnhart implements temporal aux arrays.
• Do we want to change how dtopo is done so that it can be handled directly within setaux rather than having an entirely separate apparatus?

[mandli]

Working on a simplified example that demonstrates the problem.

[mandli]

Some additional notes:

• Does not manifest until there are at least 2 levels.
• A fixed grid does not solve the problem
• Gauges do not need to be near patch boundaries
  The example above uses a max1d=500 and a fixed grid around the gauges with level=2 and produces the problem. The topography and bathymetry stored in the aux array does not demonstrate the problem.

[mandli]

It appears that at a regridding step that the aux array boundaries are not filled in properly. Outputting every step and setting the pressure field to a large value in setaux leads to spurious waves at the grid cell boundaries. It is possible that this was always a problem, but the grid boundaries and the values placed in them may have been similar enough to the true values that it was not obvious.

[mandli]

Through a process of elimination we realized that while qad is involved (turning it off fixed this) that it was because qad was getting not getting the updated values that b4step would be providing and then occasionally leading to bad values. Moving b4step out of stepgrid and calling it before qad fixes this. Still in testing to see if this breaks anything. Note that due to the bug this is also present in AMRClaw and can be replicated in the 2D advection swirl example.

[rjleveque]

@mandli, @mjberger -- thanks for digging into this. It sounds like moving the call to b4step might solve the problem, which would be great.

Just a comment on making setaux time dependent -- it's probably best to avoid this if we can since for many problems setaux only needs to be called when regridding, and not at intermediate times. So it might be misleading if the user thinks it is possible to set time-dependent aux values in setaux and have them properly adjusted every timestep. That's why the amrclaw/examples/2d_advection_swirl example calls setaux from b4step so that the time-dependent (and spatially varying) advection velocities are properly set before every time step. But apparently not always properly, due to this bug, so great if it can be fixed.