Pickle of object with getattr method in Python returns `TypeError, object not callable`

Question

I wanted to define a class that returns None for unknown attributes with __getattr__ method.

After doing that, I am trying to dump an object of that class to a Pickle.

However, I got the error

Traceback (most recent call last):
  File "c:\SVN\Scripts\Rally\examples\t_pickle_None.py", line 14, in <module>
    pickle.dump(toto, f, pickle.HIGHEST_PROTOCOL)
TypeError: 'NoneType' object is not callable

Without defining __getattr__, it works fine, but I would like to keep this function.

Here is my code: how to make it work with __getattr__?

Thanks

import pickle
from typing import Any

class Toto:
    def __init__(self, name:str) -> None:
        self.name = name

    def __getattr__(self, _: str) -> Any:
        """Return None for all unknown attributes"""
        return None

toto = Toto("Toto")
with open('toto.pkl', 'wb') as f:
    pickle.dump(toto, f, pickle.HIGHEST_PROTOCOL)

You need to be careful pickling objects that define a __getattr__. I wrote something about it here. Does it help? — PM 2Ring
– PM 2Ring, Commented Jun 16, 2018 at 13:42
Also see stackoverflow.com/questions/49380224/… and the links there. — PM 2Ring
– PM 2Ring, Commented Jun 16, 2018 at 13:48
@PM2Ring Interseting share, although the answer of Aran-Fey appared much clearer (especially at midnight when eyes are half closed :) Thanks — Jean-Francois T.
– Jean-Francois T., Commented Jun 16, 2018 at 16:37
I agree that Aran-Fey's answer is good, in fact I already upvoted it. :) But I figured that those links had good relevant info, even if they didn't exactly answer your question. If they did answer your question then I would have closed it as a duplicate. :) — PM 2Ring
– PM 2Ring, Commented Jun 16, 2018 at 16:39
Indeed. I like your idea of UserDict. I tried to make it work for my other problem of mypy complaining about dynamically created outputs but mypy complains also about your code :) ... but this is a topic for another day (or more specifically another question). — Jean-Francois T.
– Jean-Francois T., Commented Jun 16, 2018 at 16:42

Aran-Fey · Accepted Answer · 2018-06-16 14:00:51Z

10

The problem is that pickle tries to check if your object has a __getstate__ method, which can be used to customize how objects are pickled. Because your class doesn't define a __getstate__ method, your __getattr__ method ends up being called and it returns None. Pickle tries to call this None value and that produces the exception you got.

There are two ways to fix this:

Define a __getstate__ and corresponding __setstate__ method in your class:

def __getstate__(self):
    return vars(self)

def __setstate__(self, state):
    vars(self).update(state)

Rewrite your __getattr__ method to handle dunder-attributes better (i.e. throwing an AttributeError instead of returning a nonsense value):

def __getattr__(self, attr: str) -> Any:
    """Return None for all unknown attributes"""
    if attr.startswith('__') and attr.endswith('__'):
        raise AttributeError
    return None

answered Jun 16, 2018 at 14:00

Aran-Fey

44.2k13 gold badges116 silver badges163 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

shoosh Over a year ago

Missing parens after AttributeError

Sunitha · Accepted Answer · 2018-06-16 13:55:36Z

2

If you override __getattr__, you have tell how exactly your object can be pickled by implementing the __getstate__ and __setstate__ methods

Have a look at here for more info

>>> import pickle
>>> class Toto:
...     def __init__(self, name:str):
...         self.name = name
...     def __getattr__(self, _: str):
...         """Return None for all unknown attributes"""
...         return None
...     def __getstate__(self): return self.__dict__
...     def __setstate__(self, d): self.__dict__.update(d)
... 
>>> 
>>> t=Toto()
>>> pickle.dumps(t)
b'\x80\x03c__main__\nToto\nq\x00)\x81q\x01}q\x02X\x04\x00\x00\x00nameq\x03X\x03\x00\x00\x00abcq\x04sb.'
>>>

answered Jun 16, 2018 at 13:55

Sunitha

12.1k2 gold badges24 silver badges23 bronze badges

Collectives™ on Stack Overflow

Pickle of object with getattr method in Python returns `TypeError, object not callable`

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related