Skip to content

FIX pickling ufuncs defined in nested modules#4800

Merged
juliantaylor merged 1 commit intonumpy:masterfrom
ogrisel:fix-ufunc-pickling
Jun 30, 2014
Merged

FIX pickling ufuncs defined in nested modules#4800
juliantaylor merged 1 commit intonumpy:masterfrom
ogrisel:fix-ufunc-pickling

Conversation

@ogrisel
Copy link
Copy Markdown
Contributor

@ogrisel ogrisel commented Jun 11, 2014

This is a fix for the following problem with ufuncs that are not defined in a top level module:

>>> from scipy.special import expit
>>> import pickle
>>> pickle.loads(pickle.dumps(expit))
Traceback (most recent call last):
  File "<ipython-input-4-c8a97e87e418>", line 1, in <module>
    pickle.loads(pickle.dumps(expit))
  File "/usr/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1133, in load_reduce
    value = func(*args)
  File "/volatile/ogrisel/envs/py27/local/lib/python2.7/site-packages/numpy/core/__init__.py", line 56, in _ufunc_reconstruct
    return getattr(mod, name)
AttributeError: 'module' object has no attribute 'expit'

Unfortunately I don't know how to write a test in numpy as all its ufuncs are importable from the namespace of the top level numpy package as far as I know.

@ogrisel ogrisel mentioned this pull request Jun 11, 2014
Closed
4 tasks
@cournape
Copy link
Copy Markdown
Member

cournape commented Jun 26, 2014

@ogrisel I think there is a test to build custom ufunc somewhere that you may want to re-use ? I don't have easy access to the code ATM.

@ogrisel
Copy link
Copy Markdown
Contributor Author

ogrisel commented Jun 26, 2014

@cournape you mean write a test with a custom ufunc generated with np.frompyfunc? That should be possible but as would need to extend it to pass a custom name="my_importable_ufunc_name" to make such python ufuncs pickle-able as well. I reading on how to use the Python C API to do that. Unfortunately I won't have time to make it work tonight. Maybe tomorrow.

@juliantaylor
Copy link
Copy Markdown
Contributor

juliantaylor commented Jun 28, 2014

you can test numpy.core.test_rational.test_add which is a testing ufunc not in the top namespace
test_ufunc.py is the right place.

@juliantaylor
Copy link
Copy Markdown
Contributor

juliantaylor commented Jun 28, 2014

also please use BUG: as the commit prefix.

@ogrisel
Copy link
Copy Markdown
Contributor Author

ogrisel commented Jun 30, 2014

@juliantaylor I added the test you suggested and rewrite the commit message.

@juliantaylor
Copy link
Copy Markdown
Contributor

juliantaylor commented Jun 30, 2014

mh more assert usage, but it was already there before ...
thanks

juliantaylor added a commit that referenced this pull request Jun 30, 2014
@juliantaylor
Merge pull request #4800 from ogrisel/fix-ufunc-pickling
b25cdd6 
FIX pickling ufuncs defined in nested modules
@juliantaylor juliantaylor merged commit b25cdd6 into numpy:master Jun 30, 2014
@ogrisel
Copy link
Copy Markdown
Contributor Author

ogrisel commented Jun 30, 2014

Thanks for the review.

@ogrisel ogrisel mentioned this pull request Apr 20, 2015
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ogrisel @cournape @juliantaylor