CODEWITHSUNDEEP
python-3.xsockets
3therk1ll
3therk1ll

Reputation: 2441

Python3 Unable to Decode POST Request Result

I am sending a POST request with sockets and trying to decode the received HTML and print to the terminal.

This works fine in my initial GET request but when I try to decode and print the POST request I just get garbled text.

How can I change my decode so that the text is readable?

POST

POST

body = "hash="+md5

headers = """\
POST / HTTP/1.1\r
Host: url.com:57555\r
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101 Firefox/66.0\r
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r
Accept-Language: en-US,en;q=0.5\r
Accept-Encoding: gzip, deflate\r
Referer: http://url.com:57555/\r
Content-Type: application/x-www-form-urlencoded\r
Content-Length: 32\r
Connection: close\r
Cookie: PHPSESSID=some_cookie\r
Upgrade-Insecure-Requests: 1\r
\r\n"""

payload = headers + body

s.sendall(payload.encode('utf-8'))
res = s.recv(4096)

print(str(res, errors='replace'))

Result...

python3 emdee5.py
HTTP/1.1 200 OK
Date: Sun, 26 May 2019 22:01:26 GMT
Server: Apache/2.4.18 (Ubuntu)
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 282
Connection: close
Content-Type: text/html; charset=UTF-8

]�1o� ���
ʒ��Ҩ��b�V��LN��؜
p�$����Py��d��FP��l�    ^�֞i�ĜmA��F7i�zd}��VͩK8}ߠ���!�n�W>�wL9ۅr�@Ȑ����� 4i��ec{"%��0���)������W���A�I��"��GD�;�܉"J��JA}x��l1��3٠.y�>Om�#5��9
                                                                                                                                           ��ڨ�p�j����JN���MQ̀)�:�p�P{K���4J^-��+�7�oV'E;'=�����l�

Upvotes: 0

Views: 64

Answers (1)

Steffen Ullrich
Steffen Ullrich

Reputation: 123639

Your request explicitly says that you are willing to accept a compressed response:

 Accept-Encoding: gzip, deflate\r

And this is therefore what you get in the response

Content-Encoding: gzip

So, the body is compressed with gzip (which explains the garbled output) and you would need to decompress it. Given that you currently don't seem to be able to properly deal with compressed responses you should not claim in your request that you support these, i.e. remove the Accept-Encoding.

Apart from that more is likely wrong with your request:

body = "hash="+md5
...
Content-Length: 32\r
...
payload = headers + body
...

Given that md5 is 32 characters hex (or 16 byte binary) the body consisting of "hash"=+md5 is likely not 32 characters long as you claim in your Content-Length. 

POST / HTTP/1.1\r

Additionally you send a HTTP/1.1 request so you have to be able to deal with chunked responses - but your code does not deal with these. 

res = s.recv(4096)

Similarly your code blindly assumes that the full response can be retrieved within a single recv which does not need to be the case.

In summary: unless you have a deeper understanding of how HTTP works (which you do not seem to have) it is recommended that you use existing libraries to handle HTTP for you, since these were written by developers who have an understanding of HTTP.
And even if you already have an understanding of HTTP you'll likely will use these libraries anyway since you'll know that HTTP is far from trivial and that it makes no sense to implement all the necessary details and edge cases by yourself in your code if something robust already exists.

Upvotes: 1

Related Questions