Constant Record Components

[ax3l]

Migrated question from @DavidSagan in marcguetg/h5particle#14

I cannot understand the Constant Record Components section of the Standard. Can someone give me a clear example? -- Thanks.

Question: What is the difference between the present formulation for Constant Record Components and the following: Say the magnetic field component B.x for one iteration is the same as the previous. So in this dataset put an attribute sameAs which is a string that has the path to the previous iteration's B.x. Doesn't this do the same thing? -- Thanks.

[ax3l]

@RemiLehe 's answer

Not really: the Constant Record Component does not correspond to constant in time but instead to constant across particles of a species (e.g. typically mass and charge) or constant across the grid (less common).

In the above case (e.g. for a quantity that is constant across all particles of a species), instead of storing this quantity as an array with one element per particle, we store the (constant) value and the shape (number of particles). This is also explained (briefly) here: http://www.openpmd.org/#/performance (use the Down arrow of your keyboard to navigate).

This can also be seen in the example dataset example-3d.tar.gz here. In any of these files, and for the species electrons the groups charge and mass do not contain any dataset, and instead the metadata of the group contains value and shape.

[ax3l]

constant across particles of a species

Exactly. Even more general, it is applied to the concept of openPMD record components, so a mesh record component can also be constant if necessary (e.g. the field of a permanent magnet).

So in this dataset put an attribute sameAs which is a string that has the path to the previous iteration's B.x

That is not necessary for temporarily constant records.

As an example, assume you have a constant set of "hall probes" around a magnet that record magnetic fields at their position over time. Just write the position once and use file format features to reference it every time you need it later on. If you choose a modern file format (e.g. HDF5) you can just use internal and external links/references to express "the same" and both openPMD and a reader's API will be fully agnostic to how it is actually stored (as a softlink).

Related question: #60

[ax3l]

So what is shape?

In the definition of constant record components, we replace a nD array with a constant attribute (value). Just for the record, we say what the extent of that constant record should be in shape because we loose that information when not writing a data set (array) which would usually be given by a file format otherwise. We just decided for shape instead of extent as a naming because it's very similar to the numpy naming.

This attribute is necessary, because otherwise one could not differentiate between a constant record component of a single scalar value and a nD array. Also, data readers might decide to actually allocate data for it while reading (so they need to know how many elements there are).