Skip to content

Conversation

@bjlittle
Copy link
Member

@bjlittle bjlittle commented Nov 16, 2023

This pull-request extends the capability of the iris_grib.message._DataProxy to detect and auto-generate missing data payload for a GRIB2 message.

The scenario is that the message data payload is all zero, and rather than serialize this within the message, thus increasing the footprint, an application can generate the data at runtime whenever the Data Representation Section [5] has bitsPerValue=0 and an empty Data Section [7].

Failure to support this extension results in no encoded Data Section [7] within a iris_grib.message._RawGribMessage causing a KeyError traceback within the user script. The impact of this is that an N-D Iris cube can be loaded but whenever at least one GRIB2 message has such missing data, touching the cube data will result in this failure.

Reference: SciTools/iris-test-data#89

@bjlittle bjlittle requested a review from pp-mo November 16, 2023 15:27
@bjlittle
Copy link
Member Author

@pp-mo As a result of this fix, we now have "unexpected successes" in the tests regarding pickling.

We could now simply not mark these tests are expected failures.

How would you like to proceed?

@bjlittle bjlittle force-pushed the fix-data-proxy-bitspervalue branch from 1d9e159 to 0537bfe Compare November 16, 2023 15:36
@bjlittle
Copy link
Member Author

Out of curiosity, I remove the @unittest.expectedFailure decorators ...

@pp-mo
Copy link
Member

pp-mo commented Nov 16, 2023

We could now simply not mark these tests are expected failures.

After some chat, I think this is OK.
I'll do some back-fill to check what the error USED to be, but I suspect we just found that latest-eccodes has fixed it (= made the pickling possible)

Copy link
Member

@pp-mo pp-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless you want to follow a tiny suggestion..
good to go I think

@pp-mo pp-mo merged commit 70f3020 into SciTools:main Nov 16, 2023
@bjlittle
Copy link
Member Author

@pp-mo Sweet, cheers 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants