Reputation: 1060
I'm having trouble understanding exactly why this is hanging. I have stripped down this example to the core components. I have a file, let's call it do_ls.py
import fabric.api
import time
host = "myhost.mydomain"
username = "username"
password = "password"
def main():
with fabric.api.settings(host_string=host,user=username,password=password):
result = fabric.api.run("ls")
if __name__ == "__main__":
main()
If I run this command: python do_ls.py
it will execute correctly. Now for the problem. I would like to run this in it's own process. So I have this file, let's call it main.py
import sys
import os
import logging
import subprocess as sp
import time
def main():
logging.basicConfig(level=logging.DEBUG)
cmd = [sys.executable, "/path/to/do_ls.py"]
p = sp.Popen(cmd, preexec_fn=os.setpgrp)
while p.poll() is None:
print "Sleeping..."
time.sleep(0.5)
print "All Done."
if __name__ == "__main__":
main()
Now if I run python main.py
this will hang forever. The problem as far as I know is that I'm running the process in a subgroup (i.e. if I take out preexec_fn=os.setpgrp
then it will work correctly). What I don't understand is, why this is the case. Especially given that the following works:
cmd = ["ssh", "-q", "username@hostname", "ls"]
p = sp.Popen(cmd, preexec_fn=os.setpgrp)
Any insight would be greatly appreciated.
Upvotes: 2
Views: 202
Reputation: 506
Since the following lines work:
cmd = ["ssh", "-q", "username@hostname", "ls"]
p = sp.Popen(cmd, preexec_fn=os.setpgrp)
but main.py
continuously hangs, I assume that
while p.poll() is None:
never evaluates to False
. So p.poll()
must always be returning None
, possibly even after the process completes. A quick search returned this conversation on Python's bug reporting site. As per that conversation, try calling sp.Popen()
with the (undocumented) _deadstate='dead'
option:
The problem is that
os.wait()
does not play nicely withsubprocess.py
.Popen.poll()
andPopen.wait()
useos.waitpid(pid, ...)
which will raise OSError if pid has already been reported byos.wait()
.Popen.poll()
swallows OSError and by default returns None.You can (sort of) fix your program by using
p.popen(_deadstate='dead')
in place ofp.popen()
. This will makepoll()
return'dead'
instead ofNone
if OSError gets caught, but this is undocumented.
Upvotes: 1