-
Notifications
You must be signed in to change notification settings - Fork 23
Description
The conversation at #365 discussed how to record the names of files that contained some or all of the data at instantiation time - typically (but not necessarily) from a read call.
I propose a new suite of original_filenames methods that allow us to define, report, and edit these "original" files - those that contained some or all of the data at instantiation time - and are independent of the get_filenames methods. get_filenames reports on any files that are used by Data objects at the time of asking, which might reported different (typically fewer) files than were being used at the time of data instantiation.
It is also the case that the daskification of the cf-python library confuses functionality of the get_filenames methods - due to the new lazy execution paradigm, the notion of files in use at the time of asking is a little fuzzy. For this reason, the get_filenames methods are likely to be deprecated. Edit: These have now been deprecated in the PR
The implementation of this will be in cfdm and entirely inherited into cf-python. See NCAS-CMS/cfdm#215 for the implementation details, which were initially proposed as (pending review):
>>> f = cfdm.read("parent_file.nc")[0]
>>> f.original_filenames()
{"parent_file.nc"}
>>> f = cfdm.read("parent_file.nc", external="external_file.nc")[0]
>>> f.original_filenames()
{"parent_file.nc", "external_file.nc"}
>>> f.original_filenames(update="file.nc")
>>> f.original_filenames()
{"parent_file.nc", "external_file.nc", "file.nc"}
>>> f.original_filenames(clear=True)
{"parent_file.nc", "external_file.nc", "file.nc"}
>>> f.original_filenames()
set()You might want to edit the original file names if you are creating constructs ab initio (without cf.read, which will set them for you automatically), or if you are creating a Field (or Data or any other construct) that is some combination of others - in this case you might want to store the superset of original files.
Many thanks to @ThibHlln for first raising this the issue in #365.