CODEWITHSUNDEEP
pythonpython-2.7socketssubprocess
Richard Clark
Richard Clark

Reputation: 251

Python subprocess.Popen() - subprocess causes sockets to remain open

I have a Python2.7 script that does some parallelism magic and finally enters Flask gui_loop. At some point a thread creates a background process with subprocess.Popen. This works.

When my script exits and if the subprocess is still running, I can't run my script again, as flask gui_loop fails with:

socket.error: [Errno 98] Address already in use

With netstat -peanut I can see the ownership of the socket transfers to the child process when the python script exits. This is how it looks when both python script and subprocess are running:

root@test:/tmp# netstat -peanut | grep 5000
tcp        0      0 127.0.0.1:5000          0.0.0.0:*               LISTEN      1000       840210      21458/python

After terminating the Python script, socket does not close but its ownership is passed to the child process: 

root@test:~/PycharmProjects/foo/gui# netstat -peanut | grep 5000
tcp        0      0 127.0.0.1:5000          0.0.0.0:*               LISTEN      1000       763103      19559/my-subprocess

Is there any way around this? The subprocess (written in C) is not doing anything on that socket and doesn't need it. Can I somehow create a subprocess without passing the gui loop socket resource to it?

I can of course terminate the process but this is not ideal as the purpose of this is to build a simple gui around some calculations and not lose the progress if the gui script happens to exit. I would have a mechanism to reattach connection to the subprocess if I just could get the gui script up and running again.

R

Upvotes: 2

Views: 2024

Answers (2)

mata
mata

Reputation: 69022

You should use close_fds=True when creating the subproces, which will cause all file descriptors (and therfore open sockets) to be closed in the child process (except for stdin/stdout/stderr).

In newer versions (python 3.2+) close_fds already defaults to True, as in most cases you don't want to inherit all open file descriptors in a child process, but in python2.7 you still need to specify it explicitly.

Upvotes: 5

Angus
Angus

Reputation: 359

You could try using the with statement. Some documentation here:

http://preshing.com/20110920/the-python-with-statement-by-example/
https://www.python.org/dev/peps/pep-0343/

This does open/close cleanup for you.

Upvotes: 1

Related Questions