Modifying metadata in model output files

[aidanheerdegen]

In some cases there are modifications to metadata for model output files that is required.

Best case would be for the model to include this metadata, but it can be difficult/impossible to enable this.

It would be good if there was a standard way to modify metadata, with a standard way to configure it.

One option is to resuscitate the addmeta project. Either call this in a userscript, or incorporate this functionality directly in payu.

[anton-seaice]

Seems related to #510

[MartinDix]

This would also be useful in rose/cylc suites so I'd like a script usable outside payu.

[aidanheerdegen]

Given Axiom uses xarray, and so can't modify netCDF attributes in-place we need another solution.

Given payu already carries a lot of the metadata around, it would be pretty simple to adapt and embed some of the code from addmeta

https://github.com/coecms/addmeta/blob/master/addmeta/addmeta.py#L67-L72

which would be fairly simple and straightforward .. however this would not satisfy:

This would also be useful in rose/cylc suites so I'd like a script usable outside payu.

So .. do we grab and adapt addmeta as a general purpose solution? Or add what we need to payu as a short-term fix and work on a more general solution that we could back-port into payu at a later date?

[aidanheerdegen]

Examples of current global metadata for ACCESS-ESM1.6

MOM5

filename = "ocean-2d-ht.nc" ;
title = "ACCESS-ESM_CMIP6" ;
grid_type = "regular" ;
grid_tile = "N/A" ;

CICE5

title = "sea ice model output for CICE" ;
contents = "Diagnostic and Prognostic Variables" ;
source = "Los Alamos Sea Ice Model (CICE) Version 5" ;
comment = "This year has 366 days" ;
comment2 = "File written on model date 07920201" ;
comment3 = "seconds elapsed into model date:      0" ;
conventions = "CF-1.0" ;
history = "This dataset was created on 2025-05-23 at 22:04:43.3" ;
io_flavor = "io_netcdf" ;

UM

history = "File atmosphere/aiihca.pa72jan converted with /g/data/vk83/apps/base_conda/envs/payu-1.1.6/lib/python3.10/site-packages/um2nc/um2netcdf.py 1.1.0 at 2025-07-08 15:39:51" ;
Conventions = "CF-1.6" ;
source = "Data from Met Office Unified Model" ;
um_version = "7.3" ;

[aidanheerdegen]

It has been been too long since I touched this .. but here are some notes I made that perhaps are useful to kick-start this again. Ping @joshuatorrance

Follow ACDD-1.3 where appropriate

Attribute Name	Description	Example/ comment	Source	CI Check
id	payu experiment UUID		metadata.yaml
license	Data license.	SPDX identifier, e.g. “CC-BY-4.0"  default license	metadata.yaml
run_id	Git commit hash of run		git repo
date_created	Date file was created	ISO 8601:2004 extended date format	file ctime
model	Model name	ACCESS-ESM1.6	metadata.yaml
title		Need to define our standard (and from what info it is formed)  Options on this one:  Stick with what payu already puts (branch name +uuid...) Come up with standard convention for “model name + config” or similar.
realm	realm(s) where variable is defined	CMIP6 examples: “atmos”, “ocean”, “atmosChem”  Intake uses CMIP6 CVs plus others – need to include in spec.	inferred from path
model_version	Model version	e.g., 2025.06.001 or ‘unknown’ if not one of our releases	Yes	Yes
base_configuration	Base configuration used	release-preindustrial+concentrations-2.0	No
experiment_url	Git repository url that describes the experiment		No	Yes
frequency	sampling frequency	CMIP examples: “mon”, “day”, “6hr” etc  In the intake yamls it’s a list of frequencies – do we follow that when it happens?   Are there non-CMIP ones to include?	No
conventions	Convention(s) used and version	“CF-1.11”	No	CFcheck
contact	Provides details about who to contact regarding this data.	TBC – form using LDAP lookup info? Again, users can disable if preferred. Default to using contact + email from metadata.yaml	metadata.yaml
data_specification	Used to provide detail on the version of the data specification used.	TBC – list name and version number at absolute minimum, would be better to have DOI to data spec repo.	No

Not currently available:

Attribute Name	Description	Example/ comment	Source	CI Check
grid	Briefly describes output grid characteristics	PID/link to grid information (code/documentation)		Not currently

Current global metadata:

MOM5

:filename = "ocean-2d-ht.nc" ;
:title = "ACCESS-ESM_CMIP6" ;
:grid_type = "regular" ;
:grid_tile = "N/A" ;

CICE5

:title = "sea ice model output for CICE" ;
:contents = "Diagnostic and Prognostic Variables" ;
:source = "Los Alamos Sea Ice Model (CICE) Version 5" ;
:comment = "This year has 366 days" ;
:comment2 = "File written on model date 07920201" ;
:comment3 = "seconds elapsed into model date:      0" ;
:conventions = "CF-1.0" ;
:history = "This dataset was created on 2025-05-23 at 22:04:43.3" ;
:io_flavor = "io_netcdf" ;

UM

:history = "File atmosphere/aiihca.pa72jan converted with /g/data/vk83/apps/base_conda/envs/payu-1.1.6/lib/python3.10/site-packages/um2nc/um2netcdf.py 1.1.0 at 2025-07-08 15:39:51" ;
:Conventions = "CF-1.6" ;
:source = "Data from Met Office Unified Model" ;
:um_version = "7.3" ;