-
Notifications
You must be signed in to change notification settings - Fork 23
Description
netCDF4.createVariable has a chunk_cache keyword that allows for the setting of the amount of memory (in bytes) used in each variable's chunk cache at the HDF5 level (https://unidata.github.io/netcdf4-python/#netCDF4.Dataset.createVariable), and it would be useful to expose this to cf.
Currently the netCDF4 default value of 16777216 bytes (i.e. 16 MiB) is used.
Changing this has no effect on the new netCDF-4 file on disk, but may be used to prevent the available memory from filling up when a very large number of netCDF-4 variables are being created. For instance, if 1024 netCDF-4 variables are being created, then by default 17179869184 bytes (i.e. 16 GiB) of memory will be needed for their chunk caches, and if this is too much then the chunk cache should be reduced.
This will be largely implemented in cfdm==1.12.2.0 by NCAS-CMS/cfdm#329