CODEWITHSUNDEEP
pythonflasksubprocess
wsg
wsg

Reputation: 161

How can I start a subprocess in Python that runs independently and continues running if the main process is closed?

I have a small Flask API that is receiving requests from a remote server. Whenever a request is received, a subprocess is started. This subprocess is simply executing a second Python file that is in the same folder. This subprocess can run for several hours and several of these subprocesses can run simultaneously. I am using stdout to write the output of the python file into a text file.

All of this is working fine, but every couple of weeks it happens that the Flask API becomes unresponsive and needs to be restarted. As soon as I stop the Flask server, all running subprocesses stop. I would like to avoid this and run each subprocess independently from the flask API.

This is a small example that illustrates what I am doing (this code is basically inside a method that can be called through the API)

import subprocess

f = open("log.txt","wb")
subprocess.Popen(["python","job.py"],cwd = "./", stdout = f, stderr = f)

I would like to achieve that the subprocess keeps running after I stop the Flask API. This is currently not the case. Somewhere else I read that the reason is that I am using the stdout and stderr parameters, but even after removing those the behavior stays the same.

Any help would be appreciated.

Upvotes: 4

Views: 3592

Answers (2)

Kaushal Pahwani
Kaushal Pahwani

Reputation: 474

Use fork() to create a child process of the process in which you are calling this function. On successful fork(), it returns a zero for the child id.

Below is a basic example of fork, which you can easily incorporate in your code.

import os
pid = os.fork()
if pid == 0:  # new process
    os.system("nohup python ./job.py &")

Hope this helps!

Upvotes: 0

Viktor Petrov
Viktor Petrov

Reputation: 444

Your sub-processes stop because their parent process dies when you restart your Flask server. You need to completely separate your sub-processes from your Flask process by running your Python call in a new shell:

from subprocess import call

# On Linux:
command = 'gnome-terminal -x bash -l -c "python job.py"'
# On Windows:
# command = 'cmd /c "python job.py"'
call(command, shell=True)

This way your Python call of job.py will run in a separate terminal window, unaffected by your Flask server process.

Upvotes: 1

Related Questions