CODEWITHSUNDEEP
pythonlinux
user2027303
user2027303

Reputation: 707

How can I save the state of running python programs to resume later?

I am developing the machine learning analysis program which has to process the 27GB of text files in linux. Although my production system won't be rebooted very often but I need to test that in my home computer or development environment.

Now I have power failure very often so I can hardly run it continuously for 3 weeks.

My programs reads the files, applies some parsing, saves the filtered data in new files in dictionary, then I apply the algorithm on those files then saves result in mysqlDB.

I am not able to find how can I save the algorithm state.

Upvotes: 1

Views: 2112

Answers (2)

Dhruv Shah
Dhruv Shah

Reputation: 130

Since the entire algorithm state can be saved in a class, you might want to use pickle (as mentioned above), but pickle comes with it's own overloads and risks.

For better ways to do the same, you might want to check out this article, which explains why you should use the camel library instead of pickle.

Upvotes: 1

tripplet
tripplet

Reputation: 335

I everything regarding the algorithm state is saved in a class, you can serialize the class an save it to disk: http://docs.python.org/2/library/pickle.html

Upvotes: 2

Related Questions