CODEWITHSUNDEEP
pythonfortrangfortran
Jose
Jose

Reputation: 2129

subprocess.Popen and buffered process output

From inside python code, I want to run a binary program that gets its parameters from stdin. Using the subprocess module, this should be straightforward:

import subprocess
command = [ 'my_program' ]
p = subprocess.Popen( command,  \
        stdin = subprocess.PIPE, stdout = subprocess.PIPE, \
        env={ "GFORTRAN_UNBUFFERED_ALL": "1"} )
p.stdin.write ( stdin_stuff )
while True:
  o = p.stdout.readline()
  if p.poll() != None: 
    break
  # Do something with stdout

Now, this launches the program, but the python script just hangs there. I understand that this may well be due gfortran (which I use to compile my_program is buffering its stdout stream. gfortran allows one to use the GFORTRAN_UNBUFFERED_ALL environmental variable, as I have done, as well as using the FLUSH() intrinsic in the fortran code, but still no luck: the python code still hangs.

Upvotes: 2

Views: 1415

Answers (2)

Sven Marnach
Sven Marnach

Reputation: 602645

To complement Aphex's answer, here the relevant part of the documentation:

Warning

Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

Upvotes: 2

Aphex
Aphex

Reputation: 7500

You should have better luck using Popen.communicate() to send strings to the process' stdin rather than manually writing to it.

stdoutdata, stderrdata = p.communicate(stdin_stuff)

Upvotes: 4

Related Questions