CODEWITHSUNDEEP
pythoncommand-linepickle
exceed
exceed

Reputation: 489

Loading an unknown amount of pickled objects in python

I have a small and simple movie registration app that lets a user register a new movie in the registry. This is currently only using pickled objects and saving the objects is not a problem but reading an unknown number of pickled objects from the file seems to be a little more complicated since i cant find any sequence of objects to iterate over when reading the file.

Is there any way to read an unknown number of pickled objects from a file in python (read into an unknown number of variables, preferably a list) ?

Since the volume of the data is so low i dont see the need to use a more fancy storage solution than a simple file.

When trying to use a list with this code:

film = Film(title, description, length)
film_list.append(film)
open_file = open(file, "ab")
try:
  save_movies = pickle.dump(film_list, open_file)
except pickle.PickleError:
  print "Error: Could not save film to file."

it works fine and when i load it i get a list returned but no matter how many movies im registering i still only get one element in the list. When typing len(film_list) it only returns the first movie that was saved/added to the file. When looking at the file it does contain the other movies that were added to the list but they are not being included in the list for some strange reason.

I'm using this code for loading the movies:

open_file = open(file, "rb")
try:
  film_list = pickle.load(open_file)
  print type(film_list) # displays a type of list
  print len(film_list) # displays that only 1 element is in the list
  for film in film_list: # only prints out one list item
    print film.name
except pickle.PickleError:
  print "Error: Unable to load one or more movies."

Upvotes: 1

Views: 2241

Answers (1)

Mike McKerns
Mike McKerns

Reputation: 35217

You can get an unknown amount of pickled objects from a file by repeatedly calling load on a file handle object.

>>> import string
>>> # make a sequence of stuff to pickle          
>>> stuff = string.ascii_letters
>>> # iterate over the sequence, pickling one object at a time
>>> import pickle
>>> with open('foo.pkl', 'wb') as f:
...     for thing in stuff:
...         pickle.dump(thing, f)
... 
>>> 
>>> things = []
>>> f = open('foo.pkl', 'rb')
>>> # load the first two objects
>>> things.append(pickle.load(f))
>>> things.append(pickle.load(f))
>>> # get the remaining pickled items
>>> while True:
...     try:          
...         things.append(pickle.load(f))
...     except EOFError:
...         break
... 
>>> stuff 
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
>>> things
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z']
>>> f.close()

Upvotes: 2

Related Questions