Skip to content

Aggregate crashes when fields contain the actual_range attribute #764

@Mark-D-Smith

Description

@Mark-D-Smith

I have been trying to load some monthly files containing air temperature data (from chess-met https://catalogue.ceh.ac.uk/documents/835a50df-e74f-4bfb-b593-804fd61d5eab; although the data themselves are not important for this error) using cf.load, with the intention of aggregating along the time axis. I get a crash from cf.aggregate when the files contain the actual_range attribute AND there are at least three files to aggregate.

This occurs because unlike some other attributes actual_range is not handled specially by cf.aggregate. The default behviour is to convert both to strings and append one to the other:

cf-python/cf/aggregate.py

Lines 4871 to 4881 in 7d78eac

if concatenate:
if value1 is not None:
if value0 is not None:
parent0.set_property(
prop, f"{value0} :AGGREGATED: {value1}"
)
else:
parent0.set_property(prop, f" :AGGREGATED: {value1}")
else:
if value0 is not None:
parent0.del_property(prop)

When the third file is appended, the string comparison is performed between a two-element np.ndarray from the new file and a string from the previous aggregation

cf-python/cf/aggregate.py

Lines 4864 to 4869 in 7d78eac

# Still here?
if isinstance(value0, str) or isinstance(value1, str):
if value0 == value1:
continue
elif parent0._equals(value0, value1):
continue
. This raises an exception as an if statement requires a scalar logical (i.e. any or all).

Since the valid_range attribute is defined in the CF-standard (https://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/build/ch02s05.html.) this should be handled. A simple correction would be to check if the property being updated is actual_range, and if so to set the parent's corresponding attribute to the minima and maxima of the two aggregated datasets. For instance:

        if prop in ("actual_range"):
            parent0.set_property(prop,[min(value0[0],value1[0]),max(value0[1],value1[1])])
            continue 

Alternatively it could be fixed by ignoring this attribute, as is currently done for valid_range & similar attributes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    aggregationRerlating to metadata-based field and domain aggregationbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions