conventions for anomaly data

[JonathanGregory]

See issue #582 for discussion of these changes.

Release checklist

• history.adoc up to date?
• Conformance document up to date?

[sethmcg]

Suggested revised text for section 7.5:

An "anomaly" is the difference between a physical quantity and its statistical norm. For example, a commonly-used anomaly is the temperature at a specific time and place minus the long-term global average temperature.

CF offers two conventions for describing anomaly data. In Section 7.5.3, "Temporal anomalies using anomaly standard names", we describe a simple convention that depends on special standard names. It can only be used for simple temporal anomalies, and is insufficient for some use-cases. In the remainder of this section and the following two (Section 7.5.1, "Anomalies with respect to a norm data variable" and Section 7.5.2, "Anomalies with respect to a norm metadata variable"), we describe two more general conventions for anomalies that can handle more complex use-cases.

The generalized definition of an anomaly value A of some physical quantity q is the difference P - N between a particular value P of q and a normal value or norm N of q. N is some statistic calculated from the values of q that lie within specified ranges of one or more of its coordinates. P can be, but is not necessarily, one of the set of values from which N is calculated.

In the same way, a data variable A containing anomalies with respect to a norm is notionally the difference between a data variable P containing the original data and a data variable N containing the statistical norm. P is typically not present in the dataset, and N is usually absent as well. The three variables have matching dimensions. P has all the same dimensions and coordinate variables as A, but N shares only a subset of the dimensions and coordinate variables of A. The other dimensions of N are the ones over which the norm is calculated.

The commonest kind of anomaly A is a "temporal anomaly": the difference between the value P of a quantity and the mean N of the same quantity over some range of time coordinates, usually called the "climatological normal", the "climate normal", or the "climatology". N is most often either a time-mean over a continuous period of multiple years or a climatological time-mean (Section 7.4, "Climatological Statistics"). The time coordinate of the anomaly may or may not lie within the range of times from which N is calculated. N has all the same dimensions and coordinate variables as A except for time.

The general convention described here is for anomalies with respect to a statistical norm calculated from any single dimension or combination of dimensions; moreover, the norm statistic does not have to be a mean. For example, anomalies might be calculated (as a function of longitude) with respect to the zonal mean, or (as a function of horizontal location) with respect to the minimum value in the area. In these examples, the norm is the zonal mean or the area minimum, respectively. When temporal anomalies are described following the convention of this section, more information can be recorded about the norm than when following the convention of Section 7.5.3, "Temporal anomalies using anomaly standard names".

[cont'd]

[sethmcg]

Under this convention, a data variable is described as an anomaly by including in its cell_methods attribute a string of the form "name: [name: …​] anomaly wrt norm". This cell_methods entry tells us:

• that the data variable contains anomalies (A in the notation of the start of this section) calculated from data (P) with the same dimensions, as the difference from the normal statistic named norm (N).
• at which stage of data processing the anomaly A = P - N was calculated. This is known because the entries in cell_methods appear in order of application.
• that N was calculated from the variation of P over dimensions identified by the name(s).

Each name must be the name of an axis (i.e., a dimension and its corresponding coordinate variable, or a scalar coordinate variable) of the anomaly data variable A. We call these the "anomaly axes". They must have standard_name attributes. Usually there is only one anomaly axis, and usually it is a spatiotemporal axis.

For instance, for a data variable containing anomalies with respect to the zonal mean, name identifies longitude as the anomaly axis e.g. "longitude: anomaly wrt norm". For an anomaly with respect to the area minimum, we need two names e.g. "lat: lon: anomaly wrt norm". As described in Section 7.3, "Cell Methods", the combination of horizontal coordinate variables can alternatively be represented by the word area, thus "area: anomaly wrt norm". For an anomaly with respect to a statistic computed over time or climatological time, name identifies the time axis of the anomaly data variable.

For each anomaly axis (although usually there is only one), N has a coordinate variable or scalar coordinate variable that indicates the range(s) of coordinates over which the statistic N was calculated from the variation of P. We call these (scalar) coordinate variables the "norm coordinate variables". For instance, if N contains zonal means, it has a norm coordinate variable for longitude that indicates the range of longitudes over which the zonal mean was calculated.

There are two alternatives for norm. In both cases, norm is an ancillary variable of the anomaly data variable (Section 3.4, "Ancillary Data").

[cont'd]

[sethmcg]

If the data variable containing the norm N (the "norm data variable") is present in the dataset, it can be named as norm in the cell_methods of the anomaly variable. Such would be the case in a dataset that contains both zonal or long-term means and anomalies relative to those means. No modification to the metadata of N is required for it to serve as the norm for A, and N may still be treated as a data variable in its own right as well. We describe the use of a norm data variable first (Section 7.5.1, "Anomalies with respect to a norm data variable") because this case is conceptually more obvious, although it is uncommon for N to be present in the dataset.

The second alternative (Section 7.5.2, "Anomalies with respect to a norm metadata variable") is where norm in cell_methods identifies a "norm metadata variable" instead of the norm data variable. A norm metadata variable contains no data of its own, only information about the axis ranges. This method can be used regardless of whether the norm data variable is also present in the dataset. This method must be used, even if the norm data variable is present, for anomalies with respect to a statistic that depends on multivalued climatological time.

For example, the second method must be used for a timeseries of hourly mean anomalies with respect to a climatological hourly mean diurnal cycle, because the norm axis is multivalued and each anomaly value is relative to a different norm value (the one for the appropriate hour). Note that the anomaly time dimension may be larger than the norm climatological time dimension: in this example, the anomaly timeseries may be several days long, while the norm time dimension spans only one climatological day.

[cont'd]

[sethmcg]

7.5.1. Anomalies with respect to a norm data variable

In this case, the word norm in cell_methods is the name of the norm data variable, which must exist in the dataset. If either the anomaly data variable or the norm data variable has a standard_name attribute, it must not be a standard name ending in _anomaly, and if they both have standard_name attributes, they must contain the same standard name. The anomaly data variable must name the norm data variable in its ancillary_variables attribute (Section 3.4, "Ancillary Data"), as well as in cell_methods, in order to indicate the link between them.

The norm data variable must have all the same axes (dimension and coordinate variable or scalar coordinate variables) as the anomaly data variable, except for the anomaly axes. For each anomaly axis, the norm data variable must have either a scalar coordinate variable (named in its coordinates attribute), or a norm coordinate variable (and corresponding dimension), in both cases with the same standard_name as the anomaly variable's coordinate variable for that axis. The norm coordinate variable(s) must have boundary variables to indicate the coordinate ranges over which N was calculated from P.

A norm coordinate variable cannot have a dimension greater than 1. Any norm coordinate dimensions must be included among the dimensions of the anomaly data variable as well as the norm data variable; likewise, any scalar norm coordinate variables must also be named in the coordinates attribute of the anomaly data variable.

The norm data variable must have a cell_methods attribute with an entry for the norm coordinate variable (or the combination of them if more than one) to indicate how N was computed from the variation of P. For instance, the norm data variable for an anomaly with respect to a time-mean must have a coordinate variable for time, and a cell_methods attribute containing an entry for this coordinate variable.

[cont'd]

[sethmcg]

A norm data variable for a climatological statistic (in the sense of Section 7.4, "Climatological Statistics") has a norm coordinate variable that must have

• a cell_methods entry containing within and over keywords, and
• a climatology attribute to identify its boundary variable.

Example 7.15 shows how the cell_methods of the norm data variable defines the norm. By contrast, metadata following the simpler convention of Section 7.5.3, "Temporal anomalies using anomaly standard names" does not completely define the norm, as illustrated in Examples 7.20 and 7.21.

In Example 7.15, the norm coordinate variable of time has just one element. If the climatological time axis is multivalued, a norm metadata variable is required (Section 7.5.2, "Anomalies with respect to a norm metadata variable").

[cont'd]

[sethmcg]

Example 7.15. Distinguishing temporal anomalies with different kinds of norm

The anomaly data variable delta_tas(time,latitude,longitude) contains daily maxima (along the time dimension) for the 16th-19th of July, 2023 of the anomaly in air_temperature with respect to the climatological mean of 1990-2019. delta_tas is the difference between the daily maxima contained in data variable tas and the time-mean climatological_tas of the entire 30-year period 1990-2019. Note that tas would not normally be included in the dataset; it is shown here for comparison with delta_tas and climatological_tas:

  time=4;
variables:
  float delta_tas(time,latitude,longitude);
    delta_tas:standard_name="air_temperature";
    delta_tas:units="degC";
    delta_tas:units_metadata="temperature: difference";
    delta_tas:cell_methods="time: maximum time: anomaly wrt climatological_tas";
    delta_tas:coordinates="climatological_time";
    delta_tas:ancillary_variables="climatological_tas";
  float tas(time,latitude,longitude);
    tas:standard_name="air_temperature";
    tas:units_metadata="temperature: on_scale";
    tas:units="degC";
    tas:cell_methods="time: maximum";
  float climatological_tas(latitude,longitude);
    climatological_tas:standard_name="air_temperature";
    climatological_tas:units="degC";
    climatological_tas:units_metadata="temperature: on_scale";
    climatological_tas:coordinates="climatological_time";
    climatological_tas:cell_methods="climatological_time: mean";
  double time(time);
    time:standard_name="time";
    time:units="days since 2023-07-16";
    time:bounds="time_bounds";
    time:calendar="standard";
  double time_bounds(time,two);
  double climatological_time;
    climatological_time:standard_name="time";
    climatological_time:units="days since 1990-01-01";
    climatological_time:bounds="climatological_time_bounds";
    climatological_time:calendar="standard";
  double climatological_time_bounds(two);
data:
  time_bounds=0,1, 1,2, 2,3, 3,4;
  climatological_time_bounds=0,10957; // 1990-01-01, 2020-01-01

[cont'd]

[sethmcg]

The cell_methods of delta_tas records that daily maxima were calculated first, then anomalies taken with respect to the long-term mean. (Although the result would have been the same if the order of operations was reversed.)

Another possibility is that the daily anomalies are calculated with respect to the 30-year July climatological mean, contained in climatological_tas. The metadata of delta_tas is the same in both cases. The two possibilities are distinguished by the cell_methods and time coordinates of climatological_tas:

dimensions:
  time=4;
  climatological_time=1;
variables:
  float delta_tas(time,climatological_time,latitude,longitude);
    delta_tas:standard_name="air_temperature";
    delta_tas:units="degC";
    delta_tas:units_metadata="temperature: difference";
    delta_tas:cell_methods="time: maximum time: anomaly wrt climatological_tas";
    delta_tas:ancillary_variables="climatological_tas";
  float tas(time,latitude,longitude);
    tas:standard_name="air_temperature";
    tas:units="degC";
    tas:units_metadata="temperature: on_scale";
    tas:cell_methods="time: maximum";
  float climatological_tas(climatological_time,latitude,longitude);
    climatological_tas:standard_name="air_temperature";
    climatological_tas:units="degC";
    climatological_tas:units_metadata="temperature: on_scale";
    climatological_tas:cell_methods="climatological_time: mean within years
      climatological_time: mean over years";
  double time(time);
    time:standard_name="time";
    time:units="days since 2023-07-16";
    time:bounds="time_bounds";
    time:calendar="standard";
  double time_bounds(time,two);
  double climatological_time(climatological_time);
    climatological_time:standard_name="time";
    climatological_time:units="days since 1990-01-01";
    climatological_time:climatology="climatological_time_bounds";
    climatological_time:calendar="standard";
  double climatological_time_bounds(climatological_time,two);
data:
  time_bounds=0,1, 1,2, 2,3, 3,4;
  climatological_time_bounds=181,10773; // 1990-07-01, 2019-08-01, i.e. July climatology

Equivalently, the climatological_time dimension could be omitted from delta_tas and climatological_tas, with the climatological_time variable instead being identified as a scalar coordinate variable in the coordinates attributes of these data variables.

[cont'd]

[sethmcg]

7.5.2. Anomalies with respect to a norm metadata variable

In this case, the word norm in cell_methods is the the name of the norm metadata variable. This is a variable in the dataset that is not the norm data variable itself (although that may also be present in the dataset), but a variable that records the definition of the norm via the attributes cell_methods (which is mandatory) and coordinates (optional). Like a grid mapping variable, it is of arbitrary type because it contains no data. Although it is formally a data variable, CF recommends that the norm metadata variable not have any of the attributes of Appendix A, Attributes other than cell_methods and coordinates. Attributes describing the norm quantity (standard name, units, etc.) are unnecessary, because they must be the same as for the anomaly data variable. The anomaly data variable must name the norm metadata variable in its ancillary_variables attribute (Section 3.4, "Ancillary Data"), as well as in cell_methods, in order to indicate the link between them.

For each anomaly axis, the norm metadata variable must have a coordinate variable with the same standard_name as the corresponding coordinate variable on the anomaly data variable. These norm coordinate variables must be scalar or size-one, except in the case where N has a climatological time axis as described in [Section 7.4] (e.g., a monthly climatology). Scalar norm coordinate variables must be named in the coordinates attribute of both the norm metadata variable and the anomaly data variable. Size-one dimensions of the norm metadata variable must also be dimensions of the anomaly data variable. The norm metadata variable must have no coordinate variables other than the norm coordinate variables.

[cont'd]

[sethmcg]

In the case where N has a multivalued climatological time axis (such as those illustrated in Examples 7.9, 7.10, and 7.11), the norm metadata variable has, as its sole dimension, the corresponding climatological time dimension. In this case, the norm metadata variable has more than one element; in all others, it has only a single element. Regardless, it is a "dummy" variable whose data values are immaterial. In all cases, norm coordinate variables must have boundary variables that indicate the coordinate ranges over which N was calculated from P.

The norm metadata variable must have a cell_methods attribute (Section 7.3, "Cell Methods") with a single entry that indicates how N was computed from the variation of P, as in Section 7.5.1, "Anomalies with respect to a norm data variable". This attribute must refer to the dimension or scalar norm coordinate variable (or combination thereof, if more than one) of the anomaly axis.

[cont'd]

---

Note: at the end of the first sentence of the first paragraph, I changed "dimension of the anomaly coordinate variable of time" to "corresponding climatological time dimension" because I found "anomaly coordinate variable of time" very hard to parse; it sounds to me like it's referring to the time dimension of the anomaly variable, not to the time dimension of the norm metadata variable. I believe that what I've written has captured the concept correctly, but let me know if I've missed something.

[sethmcg]

The use of norm metadata variables is illustrated by Examples 7.16, 7.17 and 7.18. The treatment of multivalued climatological time is described and illustrated after Example 7.18.

Example 7.16. Temporal anomalies with a climatological norm metadata variable

This example shows how the metadata of Example 7.15 can be recorded using a norm metadata variable. The data would be the same as in that example.

A data variable delta_tas(time,latitude,longitude) contains daily maxima for 16th-19th July 2023 (along the time dimension) of the anomaly in air_temperature with respect to the climatological mean of 1990-2019.

variables:
  float delta_tas(time,latitude,longitude);
    delta_tas:standard_name="air_temperature";
    delta_tas:units="degC";
    delta_tas:units_metadata="temperature: difference";
    delta_tas:cell_methods="time: maximum time: anomaly wrt climatological_tas";
    delta_tas:coordinates="climatological_time";
    delta_tas:ancillary_variables="climatological_tas";
  float climatological_tas;
    climatological_tas:coordinates="climatological_time";
    climatological_tas:cell_methods="climatological_time: mean";
  double time(time);
    time:standard_name="time";
    time:units="days since 2023-07-16";
    time:bounds="time_bounds";
    time:calendar="standard";
  double time_bounds(time,two);
  double climatological_time;
    climatological_time:standard_name="time";
    climatological_time:units="days since 1990-01-01";
    climatological_time:bounds="climatological_time_bounds";
    climatological_time:calendar="standard";
  double climatological_time_bounds(two);

If the daily anomalies are calculated with respect to the 30-year July climatological mean:

  float delta_tas(time,latitude,longitude);
    delta_tas:standard_name="air_temperature";
    delta_tas:units="degC";
    delta_tas:units_metadata="temperature: difference";
    delta_tas:cell_methods="time: maximum time: anomaly wrt climatological_tas";
    delta_tas:coordinates="climatological_time";
    delta_tas:ancillary_variables="climatological_tas";
  float climatological_tas(climatological_time);
    climatological_tas:cell_methods="climatological_time: mean within years
      climatological_time: mean over years";
  double time(time);
    time:standard_name="time";
    time:units="days since 2023-07-16";
    time:bounds="time_bounds";
    time:calendar="standard";
  double time_bounds(time,two);
  double climatological_time(climatological_time);
    climatological_time:standard_name="time";
    climatological_time:units="days since 1990-01-01";
    climatological_time:climatology="climatological_time_bounds";
    climatological_time:calendar="standard";
  double climatological_time_bounds(climatological_time,two);

As in Example 7.15, the metadata of delta_tas is the same in the two cases. They are distinguished by the cell_methods and time coordinates of climatological_tas.

[cont'd]

[sethmcg]

Example 7.17. Anomalies with respect to a zonal mean

dimensions:
  time=100;
  latitude=180;
  longitude=360;
  zmlongitude=1;
variables:
  float rtoa(time,latitude,longitude);
    rtoa:standard_name="toa_net_downward_radiative_flux";
    rtoa:units="W m-2";
    rtoa:cell_methods="time: mean latitude: mean longitude: anomaly wrt zm";
    rtoa:coordinates="zmlongitude";
    rtoa:ancillary_variables="zm";
  float longitude(longitude);
    longitude:standard_name="longitude";
    longitude:units="degrees_E";
  float zm(zmlongitude);
    zmlongitude:cell_methods="zmlongitude: mean";
    zmlongitude:bounds="zmbounds";
  float zmlongitude(zmlongitude);
    zmlongitude:standard_name="longitude";
    zmlongitude:units="degrees_E";
  float zmbounds(zmlongitude,two);
data:
  zmlongitude=0; // value is arbitrary and immaterial
  zmbounds=-240,-90;

The zmbounds indicate that rtoa contains anomalies with respect to the zonal mean calculated from 120°E to 90°W.

[cont'd]

[sethmcg]

Example 7.18. Anomalies with respect to the minimum within a horizontal area

The data variable topography contains values of the surface altitude relative to the lowest point within a horizontal area delimited by bounds in the horizontal projection coordinates.

  float topography(y,x);
    topography:standard_name="surface_altitude";
    topography:units="m";
    topography:cell_methods="area: anomaly wrt areamin"; // or "x: y: anomaly wrt areamin"
    topography:coordinates="xrange yrange";
    topography:ancillary_variables="areamin";
    topography:grid_mapping="national_grid";
  float areamin;
    areamin:coordinates="xrange yrange";
    areamin:cell_methods="area: minimum"; // or "xrange: yrange: minimum"
  float xrange;
    xrange:standard_name="projection_x_coordinate";
    xrange:units="km";
    xrange:bounds="xbounds";
  float xbounds(two);
  float yrange;
    yrange:standard_name="projection_y_coordinate";
    yrange:units="km";
    yrange:bounds="ybounds";
  float ybounds(two);

[cont'd]

[sethmcg]

If the norm has a multivalued climatological time axis, further information must be provided to describe the correspondence between elements of the anomaly and elements of the norm. For example, in the case of an anomaly relative to a monthly climatology, all of the January anomaly values will be relative to the average value for January, all the February anomalies will be relative to the average value for February, and so on.

The mapping between the time axis of the anomaly variable and the climatological time axis of the norm is recorded by an auxiliary coordinate variable named by the coordinates attribute of the norm metadata variable. This auxiliary coordinate variable is one-dimensional and has the same time dimension as the anomaly data variable. The value of each element of the auxiliary coordinate variable is the index (numbering from 0) of the climatological time dimension of the norm corresponding to that element of the anomaly. The norm metadata variable also has the same time dimension as the anomaly data variable (rather than being a scalar), but as usual, its values are arbitrary and meaningless.

The climatological time axis of N must be included in the file, although N itself need not be present. The auxiliary coordinate variable is linked to this axis by a compress attribute naming the climatological time axis. This method of mapping between axes is equivalent to compression by gathering, described in Section 8.2, and using the same attribute for both means that the climatological time axis can be subset to only those values that are used.

In the following example, by "timestep i" we mean the slice of the data variable along its time dimension with index i (recalling that index numbering starts with 0). Suppose that A contains monthly anomalies for the months of June, July, and August of 2023 and 20240 with respect toN, the 30-year climatological monthly means for 1990—2019. N has a climatological time axis with a dimension of 12, one for each month January through December. The anomaly axis is time, and A has a time dimension of 6 (two years times three months). The first time coordinate of A is June 2023, and the first value of the auxiliary coordinate variable is 5, indicating that timestep 0 of A is the anomaly with respect to the timestep 5 of N (which is June for both). It would also be valid to use compression by gathering and provide only the climatology for June, July, and August, in which case the climatological time axis would have a dimension of 3.

In an abstract sense, the norm metadata variable indicates that A is the difference between P and N', where N' has all same dimensions as A and P, including time, and is constructed by selecting, with repetition if necessary, and concatenating the slices of N whose climatological time indices are listed in the auxiliary coordinate variable.

Example 7.19 illustrates this convention, using the example described above.

[cont'd]

---

Comment: I had a hard time understanding this section, I think because the way it was originally written presumed proficiency with compression by gathering, which I've never used. I've rewritten it pretty extensively to try to factor that out into a separate piece of the explanation, so please check that I didn't lose anything.

[sethmcg]

Example 7.19. An anomaly data variable whose norm has a multivalued climatological time coordinate variable

The anomaly data variable delta_tas contains anomalies for the months of June, July and August of 2023 and 2024 with respect to the 30-year climatological means of those months for 1990—​2019. As well as the mandatory norm metadata variable climatological_tas_metadata, we show the norm data variable climatological_tas_data and the variable tas from which the anomalies were calculated, although these are often not included in the dataset, to illustrate the relationship between them.

dimensions:
  time=6;
  climatological_time=12;
variables:
  float delta_tas(time,latitude,longitude);
    delta_tas:standard_name="air_temperature";
    delta_tas:units="degC";
    delta_tas:units_metadata="temperature: difference";
    delta_tas:cell_methods="time: maximum time: anomaly wrt climatological_tas";
    delta_tas:ancillary_variables="climatological_tas_metadata";
  float tas(time,latitude,longitude);
    tas:standard_name="air_temperature";
    tas:units="degC";
    tas:units_metadata="temperature: on_scale";
    tas:cell_methods="time: maximum";
  int climatological_tas_metadata(time);
    climatological_tas_metadata:coordinates="month_indices";
    climatological_tas_metadata:cell_methods=
      "climatological_time: mean within years
      climatological_time: mean over years";
  int month_indices(time);
    month_indices:compress="climatological_time";
  double time(time);
    time:standard_name="time";
    time:units="days since 2023-06-01";
    time:bounds="time_bounds";
    time:calendar="standard";
  double time_bounds(time,two);
  double climatological_time(climatological_time);
    climatological_time:standard_name="time";
    climatological_time:units="days since 1990-01-01";
    climatological_time:bounds="climatological_time_bounds";
    climatological_time:calendar="standard";
  double climatological_time_bounds(climatological_time,two);
  float climatological_tas_data(climatological_time,latitude,longitude);
    climatological_tas_data:standard_name="air_temperature";
    climatological_tas_data:units="degC";
    climatological_tas_data:units_metadata="temperature: on_scale";
    climatological_tas_data:cell_methods=
      "climatological_time: mean within years
      climatological_time: mean over years";
data:
  time=15, 45, 76, 381, 411, 442;
    // 2023-06-16, 2023-07-16, 2023-08-16, 2024-06-16, 2024-07-16, 2024-08-16
  time_bounds=0,30, 30,61, 61,92, 366,396, 396,427, 427,458;
    // beginning and end of Jun, Jul and Aug of 2023 and 2024
  climatological_time=15, 45, ... 349;  // 1990-01-16, 1990-02-15 ... 1990-12-16
  climatological_time_bounds=0,10623, 31,10651, ... 334,10957;
    // 1990-01-01,2019-02-01, 1990-02-01,2019-03-01 ... 1990-12-01,2020-01-01
  month_indices=5, 6, 7, 5, 6, 7;

Element 0 of month_indices is 5. This means that element 0 of time, which is 2023-06-16 (with bounds of 2023-06-01 and 2023-07-01, i.e. the whole of June 2023) is paired with element 5 of climatological_time, which is the climatology for June 1991-2000. It indicates that delta_tas(0,:,:), where by : we mean the entire range of the dimension, is the difference between tas(0,:,:) and climatological_tas(5,:,:).

[sethmcg]

7.5.3. Temporal anomalies using anomaly standard names

Several CF standard names ending with _anomaly have been defined for anomalies with respect to a long-term or climatological time-mean. The start and end of the climatological normal period may optionally be recorded in the bounds of a variable with a standard name of reference_epoch. This variable may be either a scalar coordinate variable, as in Example 7.20, or a coordinate variable with a single size-one dimension. This convention using _anomaly standard names and reference_epoch is restricted to those common cases that have such standard names defined.

Example 7.20. An anomaly data variable with a reference epoch

variables:
  float delta_tas(time,latitude,longitude);
    delta_tas:standard_name="air_temperature_anomaly";
    delta_tas:units="degC";
    delta_tas:units_metadata="temperature: difference";
    delta_tas:coordinates="reference_epoch";
    delta_tas:cell_methods="time: maximum";
  double time(time);
    time:standard_name="time";
    time:units="days since 2023-07-16";
    time:bounds="time_bounds";
    time:calendar="standard";
  double time_bounds(time,two);
  double reference_epoch;
    reference_epoch:standard_name="reference_epoch";
    reference_epoch:units="days since 1990-01-01";
    reference_epoch:bounds="reference_epoch_bounds";
    reference_epoch:calendar="standard";
  double reference_epoch_bounds(two);
data:
  time_bounds=0,1, 1,2, 2,3, 3,4;
  reference_epoch_bounds=0,10957; // 1990-01-01, 2020-01-01

The data variable delta_tas contains daily maximum temperatures for 16th-19th July 2023 expressed as anomalies with respect to a long-term time-mean. Note that delta_tas has the attribute units_metadata="temperature: difference" because it is the difference between two temperatures, as recommended in Section 3.1.2, "Temperature units". The climatological reference period is defined as 1990-2019 by the bounds of reference_epoch (10957 days since 1st January 1990 in the standard calendar is 1st January 2020). The single value of the reference_epoch coordinate variable should be a representative time within the climatological interval (see Section 7.4, "Climatological Statistics").

In this example, reference_epoch is a scalar coordinate variable. Alternatively, it could be defined with a dimension reference_epoch=1, a one-dimensional coordinate variable reference_epoch(reference_epoch) and bounds reference_epoch_bounds(reference_epoch,two). In this case, reference_epoch must be included among the dimensions of delta_tas e.g. delta_tas(reference_epoch,time,latitude,longitude), but the coordinates attribute is not needed, unlike in the case of a scalar coordinate variable.

[sethmcg]

This convention, using _anomaly standard names and reference_epoch, is simpler than using a norm data or metadata variable, but provides no information about the nature of the norm, which is described by its cell_methods attribute. The norm data variable is not necessarily present in the file, and even if it is present, this approach does not provide any link between the anomaly variable and the norm variable. Hence, interpretation of the anomaly can be unclear. Example 7.21 illustrates this ambiguity as it manifests in Example 7.20. Such ambiguities can be resolved by the conventions of Section 7.5.1, "Anomalies with respect to a norm data variable" and Section 7.5.2, "Anomalies with respect to a norm metadata variable".

Example 7.21. Ambiguity in interpreting an anomaly data variable with a reference epoch

The standard name air_temperature_anomaly of Example 7.20 is defined as the difference of air temperature from its climatology, i.e., delta_tas (A) = tas (P) - climatological_tas (N). P and N may or may not be contained in the dataset, but N is not fully described by metadata alone.

One possibility is that delta_tas is the difference between daily maxima in tas and the time-mean of the entire 30-year period 1990—​2019 in climatological_tas:

  float tas(time,latitude,longitude);
    tas:standard_name="air_temperature";
    tas:units="degC";
    tas:units_metadata="temperature: on_scale";
    tas:cell_methods="time: maximum";
  float climatological_tas(latitude,longitude);
    climatological_tas:standard_name="air_temperature";
    climatological_tas:units="degC";
    climatological_tas:units_metadata="temperature: on_scale";
    climatological_tas:coordinates="reference_epoch";
    climatological_tas:cell_methods="reference_epoch: mean";

Another possibility is that the daily anomalies are calculated with respect to the 30-year climatological mean for July:

  float climatological_tas(climmonths,latitude,longitude);
    climatological_tas:standard_name="air_temperature";
    climatological_tas:units="degC";
    climatological_tas:units_metadata="temperature: on_scale";
    climatological_tas:cell_methods="climmonths: mean within years climmonths: mean over years";

Without additional information about N, these possibilities (and others) cannot be distinguished using this convention.

(Note that in all cases, tas and climatological_tas both have the attribute units_metadata="temperature: on_scale", as recommended in Section 3.1.2, "Temperature units".)

[sethmcg]

"present in the dataset as well" sounds weird to me. I would prefer "also present in the dataset."

[JonathanGregory]

OK, changed.

[sethmcg]

Typo: "as well being" -> "as well as being"

[JonathanGregory]

Thanks, fixed.

[sethmcg]

I think 'norm' should not be italicized here.

[JonathanGregory]

Yes, quite right. I got trigger-happy with italics.

[sethmcg]

I think norm should not be italicized here, either.

[JonathanGregory]

Quite so.

[sethmcg]

Nor should norm be italicized here, IMO.

[JonathanGregory]

I agree. I have checked (maybe you did too) and there are no other offending italics.

[sethmcg]

Typo: "and and"

[JonathanGregory]

Thanks

[davidhassell]

Suggested change

	double climatological_time; // norm coordinate variable
	double climatological_time; // norm and anomaly coordinate variable

[JonathanGregory]

Not done, as discussed

[davidhassell]

Sorry - I un-apologise :) The problem is in the CDL below (this comment is on the caption for that)

[TomLav]

Yes!

[davidhassell]

I'm somewhat lost with this example.

• I presume that climatological_tas should be climatological_tas_metadata, as per the caption
• The compression by gathering applies to the data variable, not the climatological_time coordinate variable
• The compressed variable's dimension is also the name of a coordinate variable (time), which causes problems when the variable is uncompressed to size 12

[davidhassell]

OK - I've absorbed some of the new rules on how this works, and it's not really compression by gathering ... but it looks like it!

[davidhassell]

I've made suggestions in compress->select above, which negates most of this comment thread, but:

• I presume that climatological_tas should be climatological_tas_metadata, as per the caption

still stands, I think.

[sethmcg]

I support using "select" instead of "compress". I see the original motivation for re-using the functionality (which is very similar), but I agree that it's better to use different names to distinguish that they have different meanings.

[JonathanGregory]

Yes, fine, thanks. Also I have corrected climatological_tas_metadata. Thanks for noticing.

[davidhassell]

I don't think that this is how compression by gathering works in chapter 8 works. Compression by gathering applies to the data of the variable that carries the compressed dimension, and also references the index variable. This seems to a brand new mechanism that allows you to cherry-pick elements from an existing coordinates.

Edit (18:18Z): I have suggested an alternative, that just entails not using "compress" (in favour of "select"), and spelling out how the selection occurs.

[JonathanGregory]

I have changed compress to select and removed the sentence comparing the mechanism to compression by gathering. The way it works is described in the previous paragraph, which I have reordered thus:

The norm metadata variable for a multivalued climatological time axis has the time dimension of the anomaly data variable as its sole dimension, and is thus itself multivalued, but its values are arbitrary and meaningless. The mapping between the anomaly time axis of the anomaly variable and the climatological time axis of the norm is recorded by an auxiliary coordinate variable of integer type named by the coordinates attribute of the norm metadata variable. The auxiliary coordinate variable is one-dimensional and has the same time dimension as the anomaly data variable. The climatological time axis of N (i.e. its dimension and coordinate variable) must be included in the file, although N itself need not be present. The auxiliary coordinate variable has a select attribute naming the climatological time dimension, in order to make the link between them. The value of element i of the auxiliary coordinate variable is the index (numbering from 0) along the climatological time dimension of the norm corresponding to element i of the anomaly time dimension. Using this method means that the norm metadata variable can refer to a subset of elements of the climatological time axis if only some of them are relevant, and it can refer repeatedly to elements of climatological time axis if there is more than one anomaly time referring to a given climatological time.

[davidhassell]

I don't think we should mandata a _FillValue attribute. The test is that it contains only missing values, and shouldn't get into the mine field of how the missing values are encoded.

[JonathanGregory]

OK. I have therefore removed the recommendation that it "not have any of the attributes of <<attribute-appendix>> other than cell_methods, coordinates and _FillValue" because it's too complicated to rephrase this including several attributes which might be relevant to missing data. I've left it as "[The norm metadata variables] does not need attributes describing the norm quantity (standard name, units, etc.), because they must be the same as for the anomaly data variable," and "its single element must indicate missing data."

[davidhassell]

As per the compress->select suggestion

Suggested change

	| <<reduced-horizontal-grid>>, <<anomalies-norm-metadata>>, <<compression-by-gathering>>
	| <<reduced-horizontal-grid>>, <<compression-by-gathering>>

[JonathanGregory]

Done.

[JonathanGregory]

Also inserted a new entry in the table:

| **`select`**
| S
| C
| <<anomalies-norm-metadata>>
| Identifies a dimension to which the values in this variable are indices.

[davidhassell]

Suggested change

	The auxiliary coordinate variable has a **`compress`** attribute naming the climatological time dimension, in order to make the link between them.
	The auxiliary coordinate variable has a **`select`** attribute naming the climatological time dimension, in order to make the link between them.

[JonathanGregory]

OK. select is a good name for it. It is not quite the same as compress anyway, because it allows repeated indices.

[davidhassell]

Suggested change

	This method of mapping between axes is equivalent to <<compression-by-gathering>>.
	The auxiliary coordinate variable implies the existence of another auxiliary coordinate variable of the same size, not in the file, whose data values are selected from the climatological time coordinate variable according to the climatological time axis index positions given in the data array.

[JonathanGregory]

I'm sorry, I'm too tired to understand this sentence, so I've done only the deletion. The function of the aux coord var is explained in the previous paragraph.

[davidhassell]

As part of the compress->select suggestion:

Suggested change

	month_indices:compress="climatological_time";
	month_indices:select="climatological_time";

[JonathanGregory]

Done

[davidhassell]

As per the compress->select suggestion

Suggested change

	| **`select`**
	| S
	| C
	| <<anomalies-norm-metadata>>
	| Identifies other dimension to which the values in this variable are indices, used for selection of climatological time coordinates for norm metadata variables.
	| **`source`**

[JonathanGregory]

Done