CODEWITHSUNDEEP
python
LostInTranslation
LostInTranslation

Reputation: 159

How to make a python instanced object reusable?

a couple of my python programs aim to

  1. format into a hash table (hence, I'm a dict() addict ;-) ) some informations in a "source" text file, and

  2. to use that table to modify a "target" file. My concern is that the "source" files I usually process can be very large (several GB) so it makes more than 10sec to parse, and I need to run that program a bunch of times. To conclude, I feel like it's a waste to reload the same large file each time I need to modify a new "target".

My thought is, if it would be possible to write once the dict() made from the "source" file in a way that python would be able to read/process much faster (I think about a format close to the one used in RAM by python), it would be great.

Is there a possibility to achieve that?

Thank you.

Upvotes: 0

Views: 187

Answers (2)

kindall
kindall

Reputation: 184161

pickle is the usual solution to such things, but if you see any value in being able to edit the saved data, and if the dictionary uses only simple types such as strings and numbers (nested dictionaries or lists are also OK), you can simply write the repr() of the dictionary to a text file, then parse it back into a Python dictionary using eval() (or, better yet, ast.literal_eval()).

Upvotes: 0

Claudiu
Claudiu

Reputation: 229341

Yea, you can marshal the dict, or you can use pickle. For the difference between the two, especially as regards to speed, see this question.

Upvotes: 4

Related Questions