Skip to content

Warn that cloudpickle is not always deterministic#9148

Merged
jrbourbeau merged 3 commits intodask:mainfrom
pavithraes:cloudpickle-deterministic-warning
Jun 6, 2022
Merged

Warn that cloudpickle is not always deterministic#9148
jrbourbeau merged 3 commits intodask:mainfrom
pavithraes:cloudpickle-deterministic-warning

Conversation

@pavithraes
Copy link
Member

If tokenize takes the cloudpickle path, it may not always get deterministic tokens. It'll be nice to warn about this if the tokenize.ensure-deterministic config is set to True.

@pavithraes pavithraes marked this pull request as draft May 31, 2022 20:15
@pavithraes pavithraes marked this pull request as ready for review June 1, 2022 11:15
@pavithraes pavithraes requested a review from ian-r-rose June 1, 2022 12:20
Copy link
Collaborator

@ian-r-rose ian-r-rose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pavithraes! The fix looks good to me, just a couple of minor suggestions on the test

Copy link
Collaborator

@ian-r-rose ian-r-rose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @pavithraes

@ian-r-rose
Copy link
Collaborator

Test failure should be fixed by #9154

@jakirkham
Copy link
Member

Restarted the failed CI job now that PR ( #9154 ) is merged

Copy link
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pavithraes -- this is in

@jrbourbeau jrbourbeau merged commit cd6a58b into dask:main Jun 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

normalize_token does not raise warning for function with non-deterministic hash

4 participants