What is the proper method to parse and send HTTP requests using sockets in Java?

Question

I'm creating a basic local proxy server, the goal is to accept http and https traffic from my web browser, parse it for information, send and receive the requests to the proper host, and then return it to the web browser.

I currently have an open socket to my web browser. I am receiving both http and https requests from the browser like so:

HTTP:

GET http://example.com/ HTTP/1.1 
Host: example.com User-Agent:
Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5 
Accept-Encoding: gzip, deflate
Connection: keep-alive 
Upgrade-Insecure-Requests: 1

HTTPS:

CONNECT example.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: example.com:443

I open a socket to the "Host:" from the above with the following code:

public void sendRequest() throws IOException{
        Socket socket = new Socket(host, port);
        //socket.getInputStream.read();
        BufferedWriter out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream(), "UTF8"));
        BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
        for(int i = 0; i < lines.size(); i++){
            out.write(lines.get(i) + "\r\n");
        }
        out.flush();
        outputReturn(in);
    }

And I receive the reply like so:

public void outputReturn(BufferedReader in){
        try{
            System.out.println("\n * Response");
            String line;
            while ((line = in.readLine()) != null) {
                System.out.println(line);
            }
        }
        catch (IOException i){
            System.out.println(i);
        }
    }

The replies come back as so:

HTTP:

* Response
HTTP/1.1 200 OK
Content-Encoding: gzip
Accept-Ranges: bytes
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Wed, 10 Apr 2019 22:53:28 GMT
Etag: "1541025663+gzip"
Expires: Wed, 17 Apr 2019 22:53:28 GMT
Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
Server: ECS (ord/4C92)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 606

;�R�TA��0
         ��W�ri]��S�V @���1k��Z��$�6���q۽���@+���l�I�I��s�PzUe���Bf
                                                                   �'��+�>���+�OF   �I4h��^@^
�ЧA�p@�M���u����������*
<�|ԅߎP���P�-�6�O��$}�Jl)ǰ_,�4yU�rQazw�r���t
                                           .�s���3�
                                                   z�_������2�Mel
                                                                 ϋ5����%�t
                                                                          뫪R���t3

��:�|�Q��]���
             V-z�|�Y3*���rKp�5th��"��C���NH����v��OOyޣ�xs�����V��$��X�6�BR�b�C��PqE���K�<�  �G�כ7����E(17Vx2�US��
%   x��)�d�����e��O&�4/䤘���~��Oi�s�X�dW�7��#�u�"��y\$]j<�L�r�˻'�ɪ�Vg?Kr {=��΋]E��^x;�ƱX
                                                                                            TU��]�[�{��s+�e����9�g���]����H�4���#�KA��'�Z�����*r�
�$�G�   ��4�n�8���㊄+c���E�hA��X���������L��RIt�[4\����

HTTPS:

CONNECT getpocket.cdn.mozilla.net:443 HTTP/1.1
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: getpocket.cdn.mozilla.net:443


 * Response
java.net.SocketException: Connection reset

Questions:

Why do I receive what seems like binary from the HTTP request?

Why so I receive nothing from my HTTPS request?

What SHOULD I be doing instead?

Thanks in advance.

Answer 1

For your HTTP request, the Content-Encoding is gzip. The binary is the gzip-compressed data.

For your HTTPS request, you're not making an SSL/TLS handshake, so the server drops the connection.

For HTTP, I don't think you need to do anything, the browser should handle it for you. There's no feasible way to proxy an HTTPS/SSL/TLS using the method you described.