CODEWITHSUNDEEP
pythonubuntusubprocess
Isaiah Y
Isaiah Y

Reputation: 99

Can I externally view the progress of my Python program running in the background?

I have a Python program that can take several days to finish running on my cloud-hosted Ubuntu machine. At it's current stage, I can only see the progress of the program only if I keep my shell open. print("processed: " + str(finalCount) + " files.") -- simple and straightforward.

Since learning of this trick, I thought that it would be great if I can check the progress of the program without having to open the shell and find the process using ps -e

Is there a simple way to achieve this?

I have a couple ideas, but I'm not sure how viable they are (since performance is paramount. These programs can run from days to weeks). The first idea would be to set up a very simple MySQL database that would be updated each time a task is finished performing. Not sure how much strain that would put on the server, however--or if this is even an efficient idea.

Second idea: have the program append it's status to a simple log file. Then if I'm curious, I can simply download the current version of the log file.

Any thoughts? Ideas? Suggestions?

Upvotes: 1

Views: 1317

Answers (3)

Paul Rougieux
Paul Rougieux

Reputation: 11399

As suggested in an answer linked to the question you cite, you can use GNU screen to keep track of a long running process. I use it frequently for long running processes and wrote a blog post with the most common command arguments.

Start a screen session with:

    screen -S sessionname

In order to find the screen session later give it a name using sessionname. Start a long running process. Then detach the session with:

    CTRL-A-D

You can re-attach the session later with:

    screen -r sessionname

In parallel, writing to a text log sounds like a good idea. Have a look at the official documentation explaining how to write log files with python.

Upvotes: 0

Ziya Tang
Ziya Tang

Reputation: 91

Python have a standard library named logging. It can record the progress while program running. Those logs can be outputted as a log file. You can try it.

Upvotes: 2

anand
anand

Reputation: 1526

Database will not be very good idea for this usecase.

Use python logging module to log the status/progress of your task, it very light-foot and won't be much of an issue if you are not updating/logging very fast.

Upvotes: 1

Related Questions