Skip to content

New "original filenames" methods #448

@davidhassell

Description

@davidhassell

The conversation at #365 discussed how to record the names of files that contained some or all of the data at instantiation time - typically (but not necessarily) from a read call.

I propose a new suite of original_filenames methods that allow us to define, report, and edit these "original" files - those that contained some or all of the data at instantiation time - and are independent of the get_filenames methods. get_filenames reports on any files that are used by Data objects at the time of asking, which might reported different (typically fewer) files than were being used at the time of data instantiation.

It is also the case that the daskification of the cf-python library confuses functionality of the get_filenames methods - due to the new lazy execution paradigm, the notion of files in use at the time of asking is a little fuzzy. For this reason, the get_filenames methods are likely to be deprecated. Edit: These have now been deprecated in the PR

The implementation of this will be in cfdm and entirely inherited into cf-python. See NCAS-CMS/cfdm#215 for the implementation details, which were initially proposed as (pending review):

>>> f = cfdm.read("parent_file.nc")[0]
>>> f.original_filenames()
{"parent_file.nc"}

>>> f = cfdm.read("parent_file.nc", external="external_file.nc")[0]
>>> f.original_filenames()
{"parent_file.nc", "external_file.nc"}
>>> f.original_filenames(update="file.nc")
>>> f.original_filenames()
{"parent_file.nc", "external_file.nc", "file.nc"}
>>> f.original_filenames(clear=True)
{"parent_file.nc", "external_file.nc", "file.nc"}
>>> f.original_filenames()
set()

You might want to edit the original file names if you are creating constructs ab initio (without cf.read, which will set them for you automatically), or if you are creating a Field (or Data or any other construct) that is some combination of others - in this case you might want to store the superset of original files.

Many thanks to @ThibHlln for first raising this the issue in #365.

Metadata

Metadata

Assignees

No one assigned

    Labels

    dataset readRelating to reading datasetsenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions