CODEWITHSUNDEEP
pythonlist
Rohith R
Rohith R

Reputation: 1329

storing and retrieving lists from files

I have a very big list of lists. One of my programs does this:

power_time_array = [[1,2,3],[1,2,3]] # In a short form

with open (file_name,'w') as out:
        out.write (str(power_time_array))

Now another independent script need to read this list of lists back.

How do I do this?

What I have tried:

with open (file_name,'r') as app_trc_file :
    power_trace_of_application.append (app_trc_file.read())

Note: power_trace_application is a list of list of lists.

This stores it as a list with one element as a huge string.

How does one efficiently store and retrieve big lists or list of lists from files in python?

Upvotes: 3

Views: 266

Answers (3)

turkus
turkus

Reputation: 4903

It will be faster:

from ast import literal_eval


power_time_array = [[1,2,3],[1,2,3]]


with open(file_name, 'w') as out:
    out.write(repr(power_time_array))


with open(file_name,'r') as app_trc_file:
    power_trace_of_application.append(literal_eval(app_trc_file.read()))

Upvotes: 0

Moinuddin Quadri
Moinuddin Quadri

Reputation: 48120

Use Json library to efficiently read and write structured information (in the form of JSON) to a text file.

Upvotes: 3

featuredpeow
featuredpeow

Reputation: 2201

You can serialize your list to json and deserialize it back. This really doesn't change anything in representation, your list is already valid json:

import json 


power_time_array = [[1,2,3],[1,2,3]] # In a short form

with open (file_name,'w') as out:
    json.dump(power_time_array, out)

and then just read it back:

with open (file_name,'r') as app_trc_file :
    power_trace_of_application = json.load(app_trc_file)

For speed, you can use a json library with C backend (like ujson). And this works with custom objects too.

Upvotes: 3

Related Questions