While investigating why I was unable to load_session when I had some ufuncs from scipy.special, I noticed that dill fails to determine the module the function belongs to depending on the order of the imports:
On a fresh interpreter session (Python 3.8.6), if you load dill prior to loading scipy.special, you get:
>>> import dill
>>> from scipy.special import gdtrix
>>> dill.dumps(gdtrix)
b'\x80\x04\x95\x1a\x00\x00\x00\x00\x00\x00\x00\x8c\x0b__mp_main__\x94\x8c\x06gdtrix\x94\x93\x94.'
Notice the incorrect __mp_main__.
Now, if you invert the ordering of the imports:
>>> from scipy.special import gdtrix
>>> import dill
>>> dill.dumps(gdtrix)
b'\x80\x04\x95$\x00\x00\x00\x00\x00\x00\x00\x8c\x15scipy.special._ufuncs\x94\x8c\x06gdtrix\x94\x93\x94.'
Oddly enough, if we call pickle.whichmodule, it finds the correct module irrespective of import order:
>>> import pickle
>>> from scipy.special import gdtrix
>>> pickle.whichmodule(gdtrix, gdtrix.__name__)
'scipy.special._ufuncs'
Or:
>>> from scipy.special import gdtrix
>>> import pickle
>>> pickle.whichmodule(gdtrix, gdtrix.__name__)
'scipy.special._ufuncs'
The only way I can think of getting an incorrect module is to pass name=None to whichmodule:
>>> pickle.whichmodule(gdtrix, None)
'__main__'
Which might indicate that some cache is initialized when the module is loaded and it somehow fails to update for some types?
While investigating why I was unable to
load_sessionwhen I had someufuncs fromscipy.special, I noticed that dill fails to determine the module the function belongs to depending on the order of the imports:On a fresh interpreter session (Python 3.8.6), if you load
dillprior to loadingscipy.special, you get:Notice the incorrect
__mp_main__.Now, if you invert the ordering of the imports:
Oddly enough, if we call
pickle.whichmodule, it finds the correct module irrespective of import order:Or:
The only way I can think of getting an incorrect module is to pass
name=Noneto whichmodule:Which might indicate that some cache is initialized when the module is loaded and it somehow fails to update for some types?