CODEWITHSUNDEEP
pythonhtmlrequestresponsehttp-1.1
Anastassis Kapetanakis
Anastassis Kapetanakis

Reputation: 183

Python socket HTTP 1.1 CONNECT request without a valid response

Well, I just want to make the following simple program that tries to create an https tunel with www.google.com at port 443. I first tried the following code:

import socket

def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect(("www.google.com", 80))
    request = "CONNECT www.google.com:443 HTTP/1.1\n\n"
    s.send(request.encode())
    print(s.recv(4096).decode())

main()

The result of that was the following:

HTTP/1.1 405 Method Not Allowed

Content-Type: text/html; charset=UTF-8

Referrer-Policy: no-referrer

Content-Length: 1592

Date: Wed, 16 Aug 2017 07:56:14 GMT

Connection: close



<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 405 (Method Not Allowed)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>405.</b> <ins>That’s an error.</ins>
  <p>The request method <code>CONNECT</code> is inappropriate for the URL <code>/</code>.  <ins>That’s all we know.</ins>

That means that the server does not allow this request to be executed. So I thought that the problem was the port number. So I changed it to 443(which is the port for https connection). The code is that:

def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect(("www.google.com", 443))
    request = "CONNECT www.google.com:443 HTTP/1.1\n\n"
    s.send(request.encode())
    print(s.recv(4096).decode())

main()

But it does not print out a valid respnse as it should have done. It gives me an empty response. The question to that is: "Why is that happening? How can I make it work properly?" Note: I don't want to use built-in urllib or urllib2 libraries. I want to do that with sockets.

Upvotes: 5

Views: 9058

Answers (1)

bakatrouble
bakatrouble

Reputation: 1844

HTTP

In your original connection to port 80 you are just using wrong Host:

import socket


def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect(('google.com', 80))
    request = b'CONNECT google.com HTTP/1.1\n\n'
    s.send(request)
    print(s.recv(4096).decode())

main()

Response:

HTTP/1.0 200 Connection established

Or use GET method right away:

request = b'GET http://google.com HTTP/1.1\n\n'

Response is the same as to HTTPS request, google.com host doesn't work for some reason.

HTTPS

You should wrap your socket in ssl tunnel (not sure if correct term) in order to connect using HTTPS, and GET method is ready to use right after connection:

import socket
import ssl


def main():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s = ssl.wrap_socket(s)
    s.connect(('google.com', 443))
    request = b'GET google.com HTTP/1.1\n\n'
    s.send(request)
    print(s.recv(4096).decode())

main()

Response:

HTTP/1.1 302 Found
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Location: https://www.google.ru/?gfe_rd=cr&ei=WwCUWc66L6qB3APs7ZPABA
Content-Length: 259
Date: Wed, 16 Aug 2017 08:20:43 GMT
Alt-Svc: quic=":443"; ma=2592000; v="39,38,37,35"

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://www.google.ru/?gfe_rd=cr&amp;ei=WwCUWc66L6qB3APs7ZPABA">here</A>.
</BODY></HTML>

Upvotes: 6

Related Questions