CODEWITHSUNDEEP
pythonsubprocesspopen
alwaysalearner
alwaysalearner

Reputation: 5

subprocess.Popen doesn't redirect output of the spawned process

I am having difficulty redirecting the output of a process created using subprocess.Popen to a file of my choice. The following is what I am doing:

#popen_test.py 
import sys, os, subprocess, time, signal

process_log = open('process.log','w')
process = subprocess.Popen([<path_to_python_binary>,"process.py"], stdout=process_log, stderr=process_log, preexec_fn=os.setsid) #launching the sub process
time.sleep(10) #waiting for sometime; this allows the sub process to print some statements. See below
process_log.flush()
os.killpg(process.pid,signal.SIGTERM)
process_log.close()

And, this is how my process.py looks. It is basically printing some statements on stdout at regular intervals:

#process.py    
import sys,os,time
print "Initially..."
while True:
    print "Hello World"
    time.sleep(1)

The process is being launched as I can see using ps, but the output is not going to the process.log--the file remains empty. Any idea what I might be doing wrong?

Upvotes: 0

Views: 597

Answers (1)

jfs
jfs

Reputation: 414865

It looks like a "block-buffering mode"-related issue. Run the script using python -u or add sys.stdout.flush() after print "hello world".

sys.stdout is fully buffered when redirected to a file by default. print "hello world" adds the string to the stdout buffer. The buffer is not flushed if you kill the subprocess and therefore nothing is written to the file if the buffer does not overflow while the process is still running (buffer size 4K-8K: os.fstat(sys.stdout.fileno()).st_blksize).

Note: the child process works with a copy of the file descriptor i.e., you can close process_log file in the parent as soon as Popen() returns -- it has no effect on the child.

Upvotes: 3

Related Questions