BUG: apply critical sections around populating the dispatch cache#27392
Merged
charris merged 2 commits intonumpy:mainfrom Sep 14, 2024
Merged
BUG: apply critical sections around populating the dispatch cache#27392charris merged 2 commits intonumpy:mainfrom
charris merged 2 commits intonumpy:mainfrom
Conversation
8aff9cd to
37ec084
Compare
Member
|
Thanks Nathan. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #27386.
This moves the locking to a higher conceptual level in the code. Now we lock the entire ufunc object whenever we go into
promote_and_get_info_and_ufuncimpl.In my testing I'm not able to reproduce the duplicate identity cache entry error @jakevdp saw in the
ml_dtypesCI locally after making this change. Unfortunately it's a multithreaded issue and inherently flaky, so I can't be 100% positive this fixes it.Also side benefit of being a lot simpler!
Updating
pythoncapi-compatlets us use the critical section macros without putting them insidePy_GIL_DISABLEDmacros. They're just open and close brace on the GIL-enabled build.One question for @seberg or maybe @mhvk: is dtype promotion ever re-entrant? I don't think so but I'd like to double-check because I think it might be problematic if we ever recursively created critical sections here.