CODEWITHSUNDEEP
pythonfilepersistence
Blorgbeard
Blorgbeard

Reputation: 103535

How to persist a data structure to a file in python?

Let's say I have something like this:

d = { "abc" : [1, 2, 3], "qwerty" : [4,5,6] }

What's the correct way to progammatically get that into a file that I can load from python later?

Can I somehow save it as python source (from within a python script, not manually!), then import it later?

Or should I use JSON or something?

Upvotes: 41

Views: 47872

Answers (7)

mhawke
mhawke

Reputation: 87134

Try the shelve module which will give you persistent dictionary, for example:

import shelve
d = { "abc" : [1, 2, 3], "qwerty" : [4,5,6] }

shelf = shelve.open('shelf_file')
for key in d:
    shelf[key] = d[key]

shelf.close()

....

# reopen the shelf
shelf = shelve.open('shelf_file')
print(shelf) # => {'qwerty': [4, 5, 6], 'abc': [1, 2, 3]}

Upvotes: 8

Jay Atkinson
Jay Atkinson

Reputation: 3287

You also might want to take a look at Zope's Object Database the more complex you get:-) Probably overkill for what you have, but it scales well and is not too hard to use.

Upvotes: 5

eric.christensen
eric.christensen

Reputation: 3241

Use the pickle module.

import pickle
d = { "abc" : [1, 2, 3], "qwerty" : [4,5,6] }
afile = open(r'C:\d.pkl', 'wb')
pickle.dump(d, afile)
afile.close()

#reload object from file
file2 = open(r'C:\d.pkl', 'rb')
new_d = pickle.load(file2)
file2.close()

#print dictionary object loaded from file
print new_d

Upvotes: 73

Roger Pate
Roger Pate

Reputation:

JSON has faults, but when it meets your needs, it is also:

  • simple to use
  • included in the standard library as the json module
  • interface somewhat similar to pickle, which can handle more complex situations
  • human-editable text for debugging, sharing, and version control
  • valid Python code
  • well-established on the web (if your program touches any of that domain)

Upvotes: 5

Eli Bendersky
Eli Bendersky

Reputation: 273796

Just to add to the previous suggestions, if you want the file format to be easily readable and modifiable, you can also use YAML. It works extremely well for nested dicts and lists, but scales for more complex data structures (i.e. ones involving custom objects) as well, and its big plus is that the format is readable.

Upvotes: 3

Miles
Miles

Reputation: 32488

Take your pick: Python Standard Library - Data Persistance. Which one is most appropriate can vary by what your specific needs are.

pickle is probably the simplest and most capable as far as "write an arbitrary object to a file and recover it" goes—it can automatically handle custom classes and circular references.

For the best pickling performance (speed and space), use cPickle at HIGHEST_PROTOCOL.

Upvotes: 16

John Kugelman
John Kugelman

Reputation: 362087

If you want to save it in an easy to read JSON-like format, use repr to serialize the object and eval to deserialize it.

repr(object) -> string

Return the canonical string representation of the object. For most object types, eval(repr(object)) == object.

Upvotes: 1

Related Questions