Skip to content

[BUG] Cannot dump proximity forest model using joblib or pickle #600

@isma3ilsamir

Description

@isma3ilsamir

Describe the bug
Cannot export trained proximity forest model locally, to be used later on with other datasets.
I tried doing so using picke and joblib but both failed.

To Reproduce

import joblib
import pickle
from sktime.classification.distance_based import ProximityForest

clf = ProximityForest(get_distance_measure= None, verbosity= 0)
clf.fit(X_train, y_train)

#### Using Pickle ####
######################
pkl_filename = "pickle_model.pkl"
with open(pkl_filename, 'wb') as file:
    pickle.dump(clf, file)

AttributeError                            Traceback (most recent call last)
~\Code\run.py in <module>
      1 pkl_filename = "pickle_model.pkl"
      2 with open(pkl_filename, 'wb') as file:
----> 3     pickle.dump(models['pforest'].clf, file)
      4

AttributeError: Can't pickle local object 'setup_all_distance_measure_getter.<locals>.pick_rand_distance_measure'


#### Using Joblib####
#####################
joblib_file = "joblib_model.pkl"
joblib.dump(clf, joblib_file)

Can't pickle <function setup_all_distance_measure_getter.<locals>.pick_rand_distance_measure at 0x000001AC07AC3EE8>: it's not found as sktime.classification.distance_based._proximity_forest.setup_all_distance_measure_getter.<locals>.pick_rand_distance_measure
Traceback (most recent call last):
  File "run.py", line 269, in <module>
    export_model(models, args)
  File "run.py", line 221, in export_model
    m.export_model(exp)
  File "~\Code\models.py", line 178, in export_model
    joblib.dump(self.clf, fname)
  File "~\.virtualenvs\Code-h9r8g-fJ\lib\site-packages\joblib\numpy_pickle.py", line 480, in dump
    NumpyPickler(f, protocol=protocol).dump(value)
  File "~\Python37\Lib\pickle.py", line 437, in dump
    self.save(obj)
  File "~\.virtualenvs\Code-h9r8g-fJ\lib\site-packages\joblib\numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "~\Python37\Lib\pickle.py", line 549, in save
    self.save_reduce(obj=obj, *rv)
  File "~\Python37\Lib\pickle.py", line 662, in save_reduce
    save(state)
  File "~\.virtualenvs\Code-h9r8g-fJ\lib\site-packages\joblib\numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "~\Python37\Lib\pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "~\Python37\Lib\pickle.py", line 859, in save_dict
    self._batch_setitems(obj.items())
  File "~\Python37\Lib\pickle.py", line 885, in _batch_setitems
    save(v)
  File "~\.virtualenvs\Code-h9r8g-fJ\lib\site-packages\joblib\numpy_pickle.py", line 282, in save
    return Pickler.save(self, obj)
  File "~\Python37\Lib\pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "~\Python37\Lib\pickle.py", line 960, in save_global
    (obj, module_name, name)) from None
_pickle.PicklingError: Can't pickle <function setup_all_distance_measure_getter.<locals>.pick_rand_distance_measure at 0x000001AC07AC3EE8>: it's not found as sktime.classification.distance_based._proximity_forest.setup_all_distance_measure_getter.<locals>.pick_rand_distance_measure

Expected behavior
To be able to dump Proximity Forest models.

Versions

Details

System:
python: 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]
executable:~.virtualenvs\Code-h9r8g-fJ\Scripts\python.exe
machine: Windows-10-10.0.18362-SP0

Python dependencies:
pip: 20.3.3
setuptools: 51.1.1
sklearn: 0.24.0
sktime: 0.5.0
statsmodels: 0.12.1
numpy: 1.19.4
scipy: 1.5.4
Cython: 0.29.21
pandas: 1.2.0
matplotlib: 3.3.3
joblib: 1.0.0
numba: 0.52.0
pmdarima: None
tsfresh: 0.17.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions