CODEWITHSUNDEEP
pythonbashsubprocess
user3313834
user3313834

Reputation: 7837

Why do we lost stdout from a terminal after running subprocess.check_output(xyz, shell=True)?

I have this bash line:

$ printf '  Number of xml files: %s\n' `find . -name '*.xml' | wc -l`
  Number of xml files: 4
$

When I run it from python in this way the python interpreter stop and my terminal does not have stdout anymore::

$ ls
input aa bb
$ python
Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
>>>
>>> import subprocess
>>> cmd = "printf  'xml files: %s\n' `find . -name '*.xml' | wc -l`"
>>> subprocess.check_output(['/bin/bash', cmd], shell=True)
$ ls  # stdout is not seen any more I have to kill this terminal
$

Obviously the question here is not how to make this bash work from python::

>>> import subprocess
>>> cmd = "printf  'xml files: %s\n' `find . -name '*.xml' | wc -l`"
>>> out = subprocess.run(cmd, shell=True, stdout=subprocess.PIPE)
>>> print(str(out.stdout, 'utf8'))
xml files: 4

>>>

The two following issues No output from subprocess.check_output() and Why is terminal blank after running python executable? does not answer the question

Upvotes: 1

Views: 401

Answers (1)

chepner
chepner

Reputation: 531225

The short version is that check_output is buffering all the output to return. When you run ls, its standard output is going to check_output's buffer, not the terminal. When you exit the shell you are currently in, you'll get all the output at once as a single Python string.

This leads to the question, why are you getting a sub shell in the first place, instead of executing the contents of cmd? First, you are using bash wrong; its argument is a file to run, not an arbitrary command line. A more correct version of what you are doing would be

cmd = "printf  'xml files: %s\n' `find . -name '*.xml' | wc -l`"
subprocess.check_output(['/bin/bash', '-c', cmd])

or, if you want subprocess to run a shell for you, instead of explicitly executing it,

subprocess.check_output(cmd, shell=True)

Combining the list argument with shell=True is almost never what you want.

Second, given your original code, check_output first tries to combine your list into a single string, which is then joined to sh -c. That is, you try to execute something like

sh -c /bin/bash "printf  'xml files: %s\n' `find . -name '*.xml' | wc -l`"

sh runs /bin/bash, and your command string is just used as an additional argument to sh which, for the purposes of this question, we can assume is ignored. So you are in an interactive shell whose standard output is buffered instead of displayed, as described in the first part of this answer.

Upvotes: 1

Related Questions