Skip to content

Allow the HDF5 "chunk cache" to be set from cfdm.write #328

Open
@davidhassell

Description

@davidhassell

netCDF4.createVariable has a chunk_cache keyword that allows for the setting of the amount of memory (in bytes) used in each variable's chunk cache at the HDF5 level (https://unidata.github.io/netcdf4-python/#netCDF4.Dataset.createVariable), and it would be useful to expose this to cfdm.

Currently the netCDF4 default value of 16777216 bytes (i.e. 16 MiB) is used.

Changing this has no effect on the new netCDF-4 file on disk, but may be used to prevent the available memory from filling up when a very large number of netCDF-4 variables are being created. For instance, if 1024 netCDF-4 variables are being created, then by default 17179869184 bytes (i.e. 16 GiB) of memory will be needed for their chunk caches, and if this is too much then the chunk cache should be reduced.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestnetCDF writeRelating to writing netCDF datasets

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions