CODEWITHSUNDEEP
pythonzlibpickle
JBWhitmore
JBWhitmore

Reputation: 12316

Using zlib and cPickle to compress/decompress a dictionary to files

I am using python to write to a file a zlib compressed and cPickled dictionary. It appears to work, however, I cannot figure out how to read the file back in.

I'm including the following code which includes several of the things I've tried (and the associated error messages). I'm getting nowhere.

import sys
import cPickle as pickle
import zlib

testDict = { 'entry1':1.0, 'entry2':2.0 }

with open('test.gz', 'wb') as fp:
  fp.write(zlib.compress(pickle.dumps(testDict, pickle.HIGHEST_PROTOCOL),9))

attempt = 0

try:
  attempt += 1
  with open('test.gz', 'rb') as fp:
    step1 = zlib.decompress(fp)
    successDict = pickle.load(step1)
except Exception, e:
  print "Failed attempt:", attempt, e

try:
  attempt += 1
  with open('test.gz', 'rb').read() as fp:
    step1 = zlib.decompress(fp)
    successDict = pickle.load(step1)
except Exception, e:
  print "Failed attempt:", attempt, e

try:
  attempt += 1
  with open('test.gz', 'rb') as fp:
    step1 = zlib.decompress(fp.read())
    successDict = pickle.load(step1)
except Exception, e:
  print "Failed attempt:", attempt, e

try:
  attempt += 1
  with open('test.gz', 'rb') as fp:
    d = zlib.decompressobj()
    step1 = fp.read()
    step2 = d.decompress(step1)
    step3 = pickle.load(step2)
except Exception ,e:
  print "Failed attempt:", attempt, e

try:
  attempt += 1
  with open('test.gz', 'rb') as fp:
    d = zlib.decompressobj()
    step1 = fp.read()
    step2 = d.decompress(step1)
    step3 = pickle.load(step2)
except Exception ,e:
  print "Failed attempt:", attempt, e

I get the following errors: 

Failed attempt: 1 must be string or read-only buffer, not file
Failed attempt: 2 __exit__
Failed attempt: 3 argument must have 'read' and 'readline' attributes
Failed attempt: 4 argument must have 'read' and 'readline' attributes
Failed attempt: 5 argument must have 'read' and 'readline' attributes

Hopefully this is just some obvious (to someone else) fix that I am just missing. Thanks for your help!

Upvotes: 9

Views: 6877

Answers (2)

Blckknght
Blckknght

Reputation: 104852

The errors you are getting on attempts 3-5 are because you're using pickle.load instead of pickle.loads. The former expects a file-like object, rather than the byte string you're getting from the decompression calls.

This will work:

with open('test.gz', 'rb') as fp:
    data = zlib.decompress(fp.read())
    successDict = pickle.loads(data)

Upvotes: 8

Drakosha
Drakosha

Reputation: 12165

According to pickle manual you need to use pickle.load*s* http://docs.python.org/release/2.5/lib/node317.html (remove the '*' obviously)

Upvotes: 3

Related Questions