ADIOS2: Flush Attributes Earlier

[ax3l]

@RemiLehe and @Yin-YinjianZhao are running WarpX backtransformed diagnostics (cc @RevathiJambunathan) with ADIOS2.

The characteristics of this I/O methods is that we open an iteration a very long time, usually most of the runtime of a simulation, and add more-and-more storeChunks to the written data.

With HDF5, they can follow the data as it is written and "just" have some undefined/invalid values in the file as the data is not yet complete. With ADIOS2, the same is in principle possible. (General note: both in HDF5 & ADIOS2, unwritten data is undefined, which is just by coincidence often zero. An analysis on the data should skip the unwritten parts.)

The problem with ADIOS2 (tested with fileBased output) right now is though, that we are not writing the attributes out until the file is closed (i.e. /openPMD). In practice, we declare all attributes early enough though, so that we could indicate/promise from the user-side at some point forward that:

• no further attribute changes and
• no new declarations of variables will be coming and
• that we only do further storeChunk calls before the pending close.

In that case, we should be able to flush the attributes when the user gives us a hint... What do you think @franzpoeschel?

[franzpoeschel]

Discussed offline: allow users to Series::lockAttributes(), guaranteeing that attributes already defined will not be overwritten. We can still define new attributes after that call.

Also, we could take the chance to support the LockWriterDefinitions call for defined variables in ADIOS2 (maybe with another call, let's not mix the two).