CODEWITHSUNDEEP
pythonjsonlistsavepickle
Alessandro
Alessandro

Reputation: 794

How to save a list in python?

I have a list composed by many matrices. I would save this list in order to read it in another script. This is my code t save my list W:

with open("My_rotation_matrix.bin", "wb") as output:
    pickle.dump(W, output)

And then to read my list in another script:

with open("My_rotation_matrix.bin", "rb") as data:
W = pickle.load(data)

The problem is that the saved My_rotation_matrix.bin is huge (1.4 GB) and when I read my list I have to wait almost 5 minutes. Is there a faster way to save/read a list in python? For example using JSON? Thanks!

Upvotes: 1

Views: 14415

Answers (1)

jfish003
jfish003

Reputation: 1332

My suggestion is if you can work with numpy. Since already have your data in a list you can just do this: 

    import numpy as np
    data = np.array(mylist)
    np.savez("Myfile", data)

Now it might take a minute or two to save this file (depending on how much RAM you have on your computer) but loading the file is rather quick, (just a few seconds for a file that was 700 MB). When you are ready to load it you can just say

    a = np.load("Myfile.npz") #make sure you use the .npz!
    b = a['arr_0']

Upvotes: 4

Related Questions