Deleting metadata and files policy and practice

[jpnavarro]

ESGF policy and implementation do not support deleting metadata or files in response to retractions.

The only exception is that if data node is no longer able to host a replica, the file replica entry may be deleted? Should we only allow flagging a replica as deleted and not actually deleting the file replica metadata?

If we permit deleting the file replica metadata for a node, do we need a method to prevent the last replica from being deleted?

[minxu74]

Currently, we do not delete any metadata for either retraction, deletion, or deprecation and only set the corresponding fields in the metadata to certain values. For their related NetCDF files, we put their names in a database. Then we could handle them in the future. If users use the ESGF supported software (Metagrid/intake-ESGF), these retracted records shall be filtered out in default. But if they use their code to directly query the public index, they can still see those records. So my suggestions is to move these files to a special directory or set the file permission not be downloadable to prevent users from getting the retracted files.