CODEWITHSUNDEEP
python-3.xpickle
user1228633
user1228633

Reputation: 551

Error loading pickled file

I have a problem loading a pickled file in python3. See the following code:

#!/usr/bin/env python3

import csv,operator
import pickle
import os.path

pathToBin = "foo/bar/foobar.bin"
pathToCSV = "foo/bar/foobar.csv" 

if os.path.isfile(pathToBin):
        print("Binary file already on harddrive, loading from there")
        transactions = pickle.loads( open( pathToBin, "rb" ))
else:
        csvIn = open(pathToCSV,'r')
        reader = csv.reader(csvIn)
        header = next(reader)
        header = header[0].split(";")

        print("Reading file")
        transactions = []
        for row in reader:
                # read file. 
        # transactions contains now lists of strings: transactions = [ ["a","b","c"], ["a2","b2","c3"], ...]
        print("Dumping python file to harddrive")
        myPickleFile = open(pathToBin,'wb')
        pickle.dump(transactions, myPickleFile, protocol=pickle.HIGHEST_PROTOCOL)

# do some more computation

Saving the file works without any problems. However, loading it gives me the following error:

transactions = pickle.loads( open( pathToBin, "rb" ))
TypeError: '_io.BufferedReader' does not support the buffer interface

Since I use python3, strings are treated differently. Thus I specifically give the "b" option for saving/loading. Anyone got an idea why this wont work?

Upvotes: 3

Views: 1909

Answers (1)

Jean-François Fabre
Jean-François Fabre

Reputation: 140168

You want to use load:

transactions = pickle.load(open( pathToBin, "rb" ))

to read from your open file handle. loads accepts bytes not a file handle (meant: load "string", now loads "bytes" after python 3 upgrade on string/bytes handling):

transactions = pickle.loads(open( pathToBin, "rb" ).read())

to read from bytes returned by your file.

I would recommend the first option for you in that case. The second option is for more complex cases.

Aside: it's better to use a with context to control when the file is closed

with open( pathToBin,"rb") as f:
    transactions = pickle.load(f)

Upvotes: 4

Related Questions