CODEWITHSUNDEEP
pythonooppickle
alphanumeric
alphanumeric

Reputation: 19329

Python Pickle Module and OOP

In the example below pickle crashes with every attempt to save an instance A to file.

Why does it happen?

How to avoid it?

How to get around it?

class Base(dict):
    def __init__(self):
        super(Base, self).__init__()

class ClassA(Base):
    def __init__(self):
        super(ClassA, self).__init__()  
    def setInstB(self, instB):
        self['instB']=instB

class ClassB(Base):
    def __init__(self):
        super(ClassB, self).__init__()
    def setInstA(self, instA):
        self['instA']=instA

instA=ClassA()
instB=ClassB()

instA.setInstB(instB)
instB.setInstA(instA)

import pickle
pickle.dump( instA, open( "save.p", "wb" ) )

Posted later.

If Base(dict) class is not declared to be a subclass of built-in dict the problem goes away. Running the code posted below raises no pickle errors. But I still want to know why pickle fails and how to make it work with Classes inheriting from dict.

class Base(object):
    def __init__(self):
        super(Base, self).__init__()

class ClassA(Base):
    def __init__(self):
        super(ClassA, self).__init__()  
    def setInstB(self, instB):
        self.instB=instB

class ClassB(Base):
    def __init__(self):
        super(ClassB, self).__init__()
    def setInstA(self, instA):
        self.instA=instA

instA=ClassA()
instB=ClassB()

instA.setInstB(instB)
instB.setInstA(instA)

import pickle
pickle.dump( instA, open( "save.p", "wb" ) )

Upvotes: 2

Views: 258

Answers (2)

Mike McKerns
Mike McKerns

Reputation: 35217

It might not be a bug in pickle, as I can pickle what you wanted to do initially without problems. If you use dill instead of pickle, then there's no issue… and all dill is is a bunch of copy_reg functions to register how to serialize different types that pickle can't handle by default.

dude@hilbert>$ python
Python 2.7.6 (default, Nov 12 2013, 13:26:39) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class Base(dict):
...     def __init__(self):
...         super(Base, self).__init__()
... 
>>> class ClassA(Base):
...     def __init__(self):
...         super(ClassA, self).__init__()  
...     def setInstB(self, instB):
...         self['instB']=instB
... 
>>> class ClassB(Base):
...     def __init__(self):
...         super(ClassB, self).__init__()
...     def setInstA(self, instA):
...         self['instA']=instA
... 
>>> instA=ClassA()
>>> instB=ClassB()
>>> 
>>> instA.setInstB(instB)
>>> instB.setInstA(instA)
>>> 
>>> import dill as pickle
>>> pickle.dump( instA, open('save.p', 'wb') )
>>> res = pickle.load( open('save.p', 'rb') ) 
>>> res
{'instB': {'instA': {...}}}
>>> res['instB']
{'instA': {'instB': {...}}}

Get dill here: https://github.com/uqfoundation

Upvotes: 1

jb.
jb.

Reputation: 23955

I think that you ran into some kind of bug in pickle. I suggest you submit it, and probably it will get fixed.

Following code can fix your problems: 

# -*- coding: utf-8 -*-
class Base(dict):
    def __init__(self):
        super(Base, self).__init__()
        pk = None

class ClassA(Base):
    def __init__(self):
        super(ClassA, self).__init__()
    def setInstB(self, instB):
        self['instB']=instB

class ClassB(Base):
    def __init__(self):
        super(ClassB, self).__init__()
    def setInstA(self, instA):
        self['instA']=instA

instA=ClassA()
instA.pk=1
instB=ClassB()
instB.pk=2

instA.setInstB(instB)
instB.setInstA(instA)

import pickle

class MyPickler(pickle.Pickler):
    def persistent_id(self, obj):
        return obj.pk

MyPickler(open( "save.p", "wb" )).dump(instA)

Pickler allows explicitly to generate unqie ids for your objects: see persistent_id method, and since these are database objects I guess that generating ids will be easy.

Upvotes: 2

Related Questions