Regression in 0.12: different exception type than PicklingError when arguments of Parallel can not be pickled #707
Description
Moved from #706 (comment). Probably because of the switch from pickle to cloudpickle.
import numpy as np
import pymc3 as pm
rng = np.random.RandomState(0)
count_data = rng.randint(1, 70, 100).astype(dtype=np.float64)
n_count_data = len(count_data)
with pm.Model() as model:
alpha = 1.0/count_data.mean() # Recall count_data is the
# variable that holds our txt counts
lambda_1 = pm.Exponential("lambda_1", alpha)
lambda_2 = pm.Exponential("lambda_2", alpha)
tau = pm.DiscreteUniform("tau", lower=0, upper=n_count_data - 1)
with model:
step = pm.Metropolis()
trace = pm.sample(5, tune=10,step=step)More context from #706 (comment):
It seems like the problem comes from the fact that pymc3 catches PickleError to fall-back to single threaded mode:
~/miniconda3/lib/python3.6/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, nuts_kwargs, step_kwargs, progressbar, model, random_seed, live_plot, discard_tuned_samples, live_plot_kwargs, compute_convergence_checks, use_mmap, **kwargs)
440 _print_step_hierarchy(step)
441 try:
--> 442 trace = _mp_sample(**sample_args)
443 except pickle.PickleError:
444 _log.warning("Could not pickle model, sampling singlethreaded.")```But now that joblib uses cloudpickle the error you get is a TypeError. Not 100% sure what's the best way to fix that, maybe catch all errors produced by cloudpickle.dump and reraise a PickleError?