CODEWITHSUNDEEP
pythoncurl
Andrey S
Andrey S

Reputation: 151

Running cURL using Python's Popen results in a different output to stdout

I have the following piece of python 2.7 code:

import sys
from subprocess import Popen, PIPE

cmd = "curl -v http://172.23.85.34 2>&1"
p = Popen(cmd, shell=True, stderr=PIPE, stdout=PIPE)
ret = p.wait()
out, err = p.communicate()
sys.stdout.write("Command is:\n" + cmd + "\nOutput:\n" + out)

For some reason, the output that I get when I run the python script, is different from the output I get when I run curl -v http://172.23.85.34 2>&1 directly from bash. What is the reason for this?

Python output:

Command is:
curl -v http://172.23.85.34 2>&1
Output:
* Rebuilt URL to: http://172.23.85.34/
*   Trying 172.23.85.34...
* TCP_NODELAY set
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 172.23.85.34 (172.23.85.34) port 80 (#0)
> GET / HTTP/1.1
> Host: 172.23.85.34
> User-Agent: curl/7.51.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 23 Mar 2017 13:35:06 GMT
< Server: Apache/2.4.23 (Unix) OpenSSL/1.0.2g
< Upgrade: h2
< Connection: Upgrade
< Last-Modified: Tue, 20 Dec 2016 09:33:57 GMT
< ETag: "61-54413bc4fb12a"
< Accept-Ranges: bytes
< Content-Length: 97
< Content-Type: text/html
<
{ [97 bytes data]
* Curl_http_done: called premature == 0
100    97  100    97    0     0   5307      0 --:--:-- --:--:-- --:--:--  5388
* Connection #0 to host 172.23.85.34 left intact
<html>
  <body>
    <h1>It works!</h1>
    <img src="Helium.jpg" alt="Helium">
  </body>
</html>

cURL output:

$ curl -v http://172.23.85.34 2>&1
* Rebuilt URL to: http://172.23.85.34/
*   Trying 172.23.85.34...
* TCP_NODELAY set
* Connected to 172.23.85.34 (172.23.85.34) port 80 (#0)
> GET / HTTP/1.1
> Host: 172.23.85.34
> User-Agent: curl/7.51.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 23 Mar 2017 13:37:01 GMT
< Server: Apache/2.4.23 (Unix) OpenSSL/1.0.2g
< Upgrade: h2
< Connection: Upgrade
< Last-Modified: Tue, 20 Dec 2016 09:33:57 GMT
< ETag: "61-54413bc4fb12a"
< Accept-Ranges: bytes
< Content-Length: 97
< Content-Type: text/html
<
<html>
  <body>
    <h1>It works!</h1>
    <img src="Helium.jpg" alt="Helium">
  </body>
</html>
* Curl_http_done: called premature == 0
* Connection #0 to host 172.23.85.34 left intact

Upvotes: 3

Views: 1341

Answers (1)

shizhz
shizhz

Reputation: 12501

There's PROGRESS METER info in the output of python script.

From man curl:

PROGRESS METER
   curl normally displays a progress meter during operations, indicating the amount of transferred data, transfer speeds and estimated time left, etc.

   curl displays this data to the terminal by default, so if you invoke curl to do an operation and it is about to write data to the terminal, it  disables  the  progress
   meter as otherwise it would mess up the output mixing progress meter and response data.

Simply speaking curl will display progress info by default, but disable it if the output is terminal, so when you run your python script, the output of curl is not terminal but piped to your python progress, makes the progress info in the output.

You'll get same output with option -s to disable it:

curl -v -s https://httpbin.org/get 2>&1

Upvotes: 2

Related Questions