Crash when passing named function as an argument #674
Description
Hello,
I am using Python 3.6.5 with joblib 0.11.1.dev0 (master).
Consider the two files:
# test_imp.py
def foo_imp(i):
return i+2# test.py
from joblib import Parallel, delayed
def foo(i):
return i+1
def wrap(i, func):
return func(i)
# runs fine
from test_imp import foo_imp
res = Parallel(n_jobs=2)([delayed(wrap)(i, foo_imp) for i in range(10)])
print(res)
# crashes
res = Parallel(n_jobs=2)([delayed(wrap)(i, foo) for i in range(10)])
print(res)When passing an imported function foo_imp as an argument, the code runs fine. For the locally defined function foo however, I get the following error:
Traceback (most recent call last):
File "/python/site-packages/joblib/externals/loky/process_executor.py", line 376, in _process_worker
call_item = call_queue.get(block=True, timeout=timeout)
File "/python3.6/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'foo' on <module 'joblib.externals.loky.backend.popen_loky_posix' from '/python/site-packages/joblib/externals/loky/backend/popen_loky_posix.py'>Is this intended behavior? It might be related to #600.