Skip to content

Conversation

@sadielbartholomew
Copy link
Member

@sadielbartholomew sadielbartholomew commented Jan 27, 2023

Towards #182, confirms that cf.aggregate works under the new "Dask way" with no modifications, with test_basic_aggregate failing instead due to specific issues elsewhere, namely:

  • an issue with data comparison by Data.equals;
  • an issue with construct comparison leading to field copies not being recognised as equal.

(Note to tidy up previous investigation/debugging branches, I created a fresh branch and re-applied commits to break down the fixes to specific parts.)

@sadielbartholomew sadielbartholomew added bug Something isn't working high priority dask Relating to the use of Dask labels Jan 27, 2023
@sadielbartholomew sadielbartholomew added this to the 3.14.0 milestone Jan 27, 2023
@sadielbartholomew sadielbartholomew self-assigned this Jan 27, 2023
@sadielbartholomew
Copy link
Member Author

sadielbartholomew commented Jan 27, 2023

@davidhassell please either hold on the reviewing here until tomorrow morning, or you can partially review under the following context:

  • as-is on this branch, test_aggregate passes fully, but notably I've commented out a set of related test assertions, which rely on a fix in cfdm for which I am preparing a PR but it is not quite ready so I will put up tomorrow morning;
  • once the corresponding cfdm PR is in, I will self-review that everything's working together and then open this for review fully.

If you like you could review the commits here standalone noting the above, with the main one is the initial tweak to fix an aspect of Data.equals, 02133f7.

@sadielbartholomew sadielbartholomew marked this pull request as ready for review January 27, 2023 16:31
@sadielbartholomew sadielbartholomew removed the bug Something isn't working label Jan 27, 2023
@sadielbartholomew
Copy link
Member Author

Thanks for all of your invaluable guidance here @davidhassell. (I manually added you as a co-author on the new fix approach commit, since you essentially wrote that code!)

This is good to review now, I think. FYI, the linting CI job is failing due to steup issues relating to GitLab (I see these on the command line too, so think it will resolve itself in time and will ignore it for now). So we can ignore that.

Copy link
Collaborator

@davidhassell davidhassell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. All makes sense following our conversations.

@sadielbartholomew sadielbartholomew merged commit 38475f9 into NCAS-CMS:master Jan 27, 2023
@sadielbartholomew sadielbartholomew deleted the lama-to-dask-aggregate-fix branch January 27, 2023 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dask Relating to the use of Dask high priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants