CODEWITHSUNDEEP
pythonsubprocess
n1k31t4
n1k31t4

Reputation: 2874

Piping command with seemingly several separate outputs to file

I am using speedtest-cli to get my internet speed in a python script. I would like to run this command in shell via subprocess.Popen.

Here is the command in terminal:

`speedtest-cli --share > output.log`

speedtest-cli runs the test, whilst --share provides me with an additional link in the output, pointing to an image of the speedtest result. Here is the content of output.log:

Retrieving speedtest.net configuration... Testing from M-net (xxx.xxx.xxx.xxx)... Retrieving speedtest.net server list... Selecting best server based on ping... Hosted by XXXXXXXXXXXXXXXXXXXXXX [16.12 km]: 20.902 ms Testing download speed................................................................................ Download: 48.32 Mbit/s Testing upload speed...................................................................................................... Upload: 12.49 Mbit/s Share results: http://www.speedtest.net/result/670483456.png

If I run the command in terminal, I get all the test results as well as the link in the target file as expected. I confirmed it is all stdout and not another channel by using this grep trick: command | grep .

I am trying to run it in Python as follows:

subprocess.Popen(['speedtest-cli', '--share', '>', 'output.log`],
                  stdout=subprocess.PIPE, shell=True)

...and I also tried putting the output into the file directly via python:

with open('output.log', 'w') as f:
    Popen(['speedtest-cli', '--json', '--share'], stdout=f, shell=True)

Neither of these work. I get a nice file created with the latter approach, BUT the link is not included! (the last line in the output above).

Against all the warnings of deprecation and better safety with using subprocess module, I became desparate and tried os.system():

os.system('speedtest-cli --share > output.log')

Annoyingly, this works... the full output along with the link is captured in the file.

What is going on here? How do I get the link to be captured using Popen?

I'm using Python 3.5

Upvotes: 0

Views: 101

Answers (1)

larsks
larsks

Reputation: 311645

When using shell=True, your argument to Popen needs to be a string, not a list:

subprocess.Popen('speedtest-cli --json --share > output.log',
                 stdout=subprocess.PIPE, shell=True)

Compare:

>>> subprocess.Popen('echo hello', shell=True)
>>> hello

And:

>>> subprocess.Popen(['echo', 'hello'], shell=True)
>>>

When you pass a list and you are using shell=True, only the the first item is relevant and the remainder is ignored.

If you want to collect the output yourself, consider subprocess.check_output:

>>> output = subprocess.check_output(['echo', 'hello'])
>>> output
b'hello\n'

Or:

>>> output = subprocess.check_output('echo hello', shell=True)

The check_output method works with both Python 2 and Python 3. In Python 3, you also have available the run method.

Upvotes: 2

Related Questions