CODEWITHSUNDEEP
pythonserializationpickle
Joey Eremondi
Joey Eremondi

Reputation: 8423

Python Pickle EOFerror when using Pickler (but not with pickle.dump())

So, I'm trying to save some objects to disk on Windows 7 using Python's pickle. I'm using the code below, which fails on pretty much any arbitrary object (the contents of saveobj aren't important, it fails regardless). Below is my test code:

import pickle, os, time
outfile = "foo.pickle"
f = open(outfile, 'wb')
p = pickle.Pickler(f, -1)
saveobj = ( 2,3,4,5,["hat", {"mat": 6}])
p.save(saveobj)
#pickle.dump(saveobj, f)
print "done pickling"
f.close()
g  = open(outfile, 'rb')
tup = pickle.load(g)
g.close()
print tup

When I run it, I get the following output/error:

done pickling
Traceback (most recent call last):
  File "C:\Users\user\pickletest2.py", line 13, in <module>
    tup = pickle.load(g)
  File "C:\Python26\lib\pickle.py", line 1370, in load
    return Unpickler(file).load()
  File "C:\Python26\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "C:\Python26\lib\pickle.py", line 880, in load_eof
    raise EOFError
EOFError

However, if I use pickle.dump() instead of a Pickler object, it works just fine. My reason for using Pickler is that I would like to subclass it so I can perform operations on each object before I pickle it.

Does anybody know why my code is doing this? My searching has revealed that not having 'wb' and 'rb' commonly cause this, as does not having f.close(), but I have both of those. Is it a problem with using -1 as the protocol? I'd like to keep it, as it can handle objects which define their own __slots__ methods without defining a __getstate__ method.

Upvotes: 1

Views: 2129

Answers (2)

Cinquo
Cinquo

Reputation: 663

In general it is better to use cPickle for performance reasons (since cPickle is written in C). Anyway, using dump it works just fine:

import pickle
import os, time
outfile = "foo.pickle"
f = open(outfile, 'wb')
p = pickle.Pickler(f, -1)
saveobj = ( 2,3,4,5,["hat", {"mat": 6}])
p.dump(saveobj)
#pickle.dump(saveobj, f)
f.close()
print "done pickling"
#f.close()
g  = open(outfile, 'rb')
u = pickle.Unpickler(g) #, -1)
tup = u.load()
#tup = pickle.load(g)
g.close()
print tup

Upvotes: 1

Thomas K
Thomas K

Reputation: 40330

Pickler.save() is a lower level method, that you're not supposed to call directly.

If you call p.dump(saveobj) instead of p.save(saveobj), it works as expected.

Perhaps it should be called _save to avoid confusion. But dump is the method described in the documentation, and it neatly matches up with the module-level pickle.dump.

Upvotes: 2

Related Questions