CODEWITHSUNDEEP
pythonfile-iodata-structures
James4701
James4701

Reputation: 63

Possible to store multiple data structures in one file for save and load (python)?

I want to write an array and a dictionary to a file (and possible more), and then be able to read the file later and recreate the array and dictionary from the file. Is there a reasonable way to do this in Python?

Upvotes: 1

Views: 2324

Answers (3)

Noufal Ibrahim
Noufal Ibrahim

Reputation: 72855

The pickle format blurs the line between data and code and I don't like using it except when I'm the only writer and reader of the data in question and when I'm sure that it's not been tampered with.

If your data structure is just non sequence types, dicts and lists, you can serialise it into json using the json module. This is a pure data format which can be read back reliably. It doesn't handle tuples though but treats them as lists.

Here's an example.

 a = [1,2,3,4]
 b = dict(lang="python", author="Guido")
 import json
 with open("data.dump", "r") as f:
    x, y = json.load(f)
 print x  # => [1, 2, 3, 4]
 print y  # =>  {u'lang': u'python', u'author': u'Guido'}

It's not totally unchanged but often good enough.

Upvotes: 0

elyase
elyase

Reputation: 41003

I recommend you use shelve (comes with python). For example:

import shelve
d = shelve.open('file.txt')           # in this file you will save your variables
d['mylist'] = [1, 2, 'a']             # thats all, but note the name for later.
d['mydict'] = {'a':1, 'b':2}
d.close()

To read values:

import shelve
d = shelve.open('file.txt')
my_list = d['mylist']           # the list is read from disk
my_dict = d['mydict']           # the dict is read from disk

If you are going to be saving numpy arrays then I recommend you use joblib which is optimized for this use case.

Upvotes: 2

user2489252
user2489252

Reputation:

Pickle would be one way to go about it (it is in the standard library).

import pickle

my_dict = {'a':1, 'b':2}

# write to file
pickle.dump(my_dict, open('./my_dict.pkl', 'wb'))

#load from file
my_dict = pickle.load(open('./my_dict.pkl', 'rb'))

And for the array you can use the ndarray.dump() method in numpy, which is more efficient for large arrays. 

import numpy as np
my_ary = np.array([[1,2], [3,4]])
my_ary.dump( open('./my_ary.pkl', 'wb'))

But you can of course also put everything into the same pickle file or use shelve (which uses pickle) like suggested in the other answer.

Upvotes: 0

Related Questions