CODEWITHSUNDEEP
pythonpickle
matt
matt

Reputation: 4249

Pickle linked objects

I want to pickle an object and a second object that references the first. When I naively pickle/unpickle the two objects, the reference becomes a copy. How do I preserve the link between the two objects foo and bar.foo_ref?

import pickle

class Foo(object):
    pass

foo = Foo()
bar = Foo()
bar.foo_ref = foo

with open('tmp.pkl', 'wb') as f:
    pickle.dump(foo, f)
    pickle.dump(bar, f)
with open('tmp.pkl', 'rb') as f:
    foo2 = pickle.load(f)
    bar2 = pickle.load(f)

print id(foo) == id(bar.foo_ref) # True
print id(foo2) == id(bar2.foo_ref) # False
# want id(foo2) == id(bar2.foo_ref)

Upvotes: 13

Views: 3530

Answers (3)

jterrace
jterrace

Reputation: 67073

My previous answer was missing your point. The problem with your code is that you're not using the Pickler and Unpickler objects. Here's a working version with multiple dump calls:

import pickle

class Foo(object):
    pass

foo = Foo()
bar = Foo()
bar.foo_ref = foo

f = open('tmp.pkl', 'wb')
p = pickle.Pickler(f)
p.dump(foo)
p.dump(bar)
f.close()

f = open('tmp.pkl', 'rb')
up = pickle.Unpickler(f)
foo2 = up.load()
bar2 = up.load()

print id(foo) == id(bar.foo_ref) # True
print id(foo2) == id(bar2.foo_ref) # True

Upvotes: 7

jterrace
jterrace

Reputation: 67073

If you pickle them together, the pickle module keeps track of references and only pickles the foo variable once. Can you pickle both foo and bar together, like this?

import pickle

class Foo(object):
    pass

foo = Foo()
bar = Foo()
bar.foo_ref = foo

with open('tmp.pkl', 'wb') as f:
    pickle.dump((foo, bar), f)
with open('tmp.pkl', 'rb') as f:
    foo2, bar2 = pickle.load(f)

print id(foo) == id(bar.foo_ref) # True
print id(foo2) == id(bar2.foo_ref) # True

Upvotes: 1

Thomas K
Thomas K

Reputation: 40340

Well, can you do:

bar2 = pickle.load(f)
foo2 = bar2.foo_ref

Let pickle handle the link for you.

Upvotes: 0

Related Questions