CODEWITHSUNDEEP
javasockets
Heidi
Heidi

Reputation: 131

Detect the end of an HTTP request (Java socket) easiest way?

I have to use Java sockets to send and receive HTTP requests (I have to!). So I'm looking for a solution that takes into account content-length and Transfer-Encoding: chunked, etc... to decide when the HTTP request has ended.

Here is an example of what I came up with so far:

    public String getWebpage()
    {
        try{

            _out.print("GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n");
            _out.flush();

            String fullRequest = "";
            String line = null;
            while (( line = _in.readLine()) != null)
            {
                fullRequest += line + "\r\n";
                if(isFullRequest(fullRequest))
                {
                    System.out.println(fullRequest);
                    return fullRequest;
                }
            }            

        }catch(Exception e){}

        return null;
    }

    private boolean isFullRequest(String request)
    {
        return request.contains("\r\n\r\n") //Make sure we have the headers
            && request.contains("</html>"); //Make sure we have the html
    }

My isFullRequest() method is a very cheap way to detect the end of the request, but isn't reliable.

Instead of reinventing the wheel and spending countless hours into debugging, I'm asking if there's already a Class or Method already included in Java that does exactly what I need ? Without extra unnecessary jar dependencies ?

Upvotes: 2

Views: 2526

Answers (2)

alain.janinm
alain.janinm

Reputation: 20065

You can use HttpUrlConnection to read a chunked response. This way you don't have to worry about detecting the end of the request.

URL url = new URL(urlStr);
HttpURLConnection uc = (HttpURLConnection)url.openConnection();
InputStream in = uc.getInputStream();
byte[] b=new byte[512*1024];
int len;
OutputStream out = new FileOutputStream(f);

while((len=in.read(b))!=-1){
   out.write(b,0,len);
}
out.flush();
out.close();
in.close();

If you want to get the content-lenght you can try :

long contentLength = uc.getContentLengthLong()

It will works only if content-length header is known.

If it's unknown there is another way (not my favourite though...). Read the stream once just to know the content-length. I used that on a project where it was mandatory to draw progress bar while downloading the content of a page.

long max = uc.getContentLengthLong();
if(max==-1){
    max=0;
    if(in.markSupported()){
        in.mark(1000000000);    //max nb of bytes to be read
        while((len=in.read(b))!=-1){
            max+=len;
        }
        in.reset();
    }
}

Upvotes: 0

Steffen Ullrich
Steffen Ullrich

Reputation: 123601

If you don't want to use an existing HTTP library it is much easier to make a HTTP/1.0 request only so that you don't get a chunked response (chunked is only defined for HTTP/1.1). Also don't use keep-alive, which is implicit with HTTP/1.1 but not with HTTP/1.0. In summary, do your request like this:

GET /page HTTP/1.0
Host: hostname

And then simply read the response until end of data. Since keep-alive is by default off with HTTP/1.0 the server will close the connection after the response is done and since HTTP/1.0 does not support chunked you don't need to worry about this too.

Upvotes: 4

Related Questions