Binary File Download using socket connection in Java ME

Question

I am trying to download a pdf file on my mobile (using Java ME) using SocketConnection Api. The idea is to send the server a HTTP GET request, and it replies back with the data for pdf file. However, the problem I am facing is that the server initially replies back with string data (the HTTP Headers), and then the binary data. I just want to store the binary data (the pdf file).

I have written this code so far, and it works perfectly fine as far as the server replies back with string data. However, when it replies back with binary data, this code still tries to store everything as string, correctly storing the initially returned HTTP Headers (not required) and then garbled bits corresponding to the binary data of my PDF file.

public void FileDownload() {
    try {
        sc = (SocketConnection) Connector.open("socket://" + hostname + ":" + port);
        OutputStream os = sc.openOutputStream();
        os.write(("GET " + link_to_file_to_be_downloaded + " HTTP/1.0\r\n").getBytes("UTF-8"));
        os.write(("HOST: " + hostname + "\r\n").getBytes("UTF-8"));
        os.write(("\r\n").getBytes("UTF-8"));
        os.flush();
        os.close();

        String url = "file:///E:/Data/" + "binary_data.pdf";
        FileConnection fconn = (FileConnection) Connector.open(url, Connector.READ_WRITE);
        if (!fconn.exists()) {
            fconn.create();
        }
        OutputStream ops = fconn.openOutputStream();
        byte data = 0;
        in = sc.openInputStream();
        data = (byte) in.read();
        while (data != -1) {
            ops.write(data);
            data = (byte) in.read();
        }
        ops.flush();
        ops.close();
        fconn.close();                

    } catch (IOException ex) {
        parent_class.main_form.append("Exception occured while "
                + "downloading file: " + ex.toString() + "\n");
    } finally {
        if (in != null) {
            try {
                in.close();
            } catch (IOException ex) {
                parent_class.main_form.append("Exception occured while "
                        + "downloading file: " + ex.toString() + "\n");
            }
        }
    }
}

This is what gets stored in the file "binary_data.pdf" using this code -

HTTP/1.1 200 OK
Date: Sun, 25 Mar 2012 07:03:10 GMT
Server: Apache/2.2.14 (Ubuntu)
Last-Modified: Tue, 20 Mar 2012 22:00:45 GMT
ETag: "420050-12bad-4bbb3ce85fd21"
Accept-Ranges: bytes
Content-Length: 76717
Content-Type: application/pdf
Via: 1.0 www.XXX.XXX.org
Connection: close

%PDF-1.4
%????
3 0 obj <<
/Length 4077      
/Filter /FlateDecode
>>
stream
x??ZYs?6~????9U.?#??Udg?M*qYJ???T-4?fq? @Z????<FT?}
lt7??n???_???4?s???????"
3????<???^?V?z??M?z??m?^????V???o??S'm6?????.??/Sx??Y?av?MB?*b^?f??/?IO??B??q??/?(??aT?a?@#??,?%???Z8? ?]??-?\?]??????nw?2?;?????Z?;?[}??????&J=ml??-??V?|??:??"?(?Gf??D??~?QW?U?Z???cP?b???QX

(This operation might be simpler using the high level HttpConnection api, but I wish to understand how everything works at the most basic level, and hence I am using the SocketConnection api instead.)

In short, what I wish my app to do is simply interpret the data replied by the server correctly, either as string or binary, and then accordingly store the binary file (possibly discarding the string HTTP headers).

Answer 1

I found the solution. Below is the working code.
I am first storing the header response as a string. Headers are terminated by \r\n\r\n, (so, read in bytes upto these characters). Later am storing the (possibly) binary data in a file separately.

    public String FileDownloadNonPersistently() {
        String server_reply = new String();
        try {
            sc = (SocketConnection) Connector.open("socket://" + hostname + ":" + port);
            os = sc.openOutputStream();
            os.write(("GET " + link_to_file_to_be_downloaded + 
                    " HTTP/1.0\r\n").getBytes("UTF-8"));
            os.write(("HOST: " + hostname + "\r\n").getBytes("UTF-8"));
            os.write(("\r\n").getBytes("UTF-8"));
            os.flush();
            os.close();

            in = sc.openInputStream();
            // 1. Read the response header from server separately beforehand.
            byte data;
            String temp_char = "";
            while (!"\r\n\r\n".equals(temp_char)) {
                data = (byte) in.read();
                server_reply += String.valueOf((char) data);
                if (((char) data) == '\r' || ((char) data) == '\n') {
                    temp_char += String.valueOf((char) data);
                } else {
                    temp_char = "";
                }
            }

            // 2. Recieving the actual data, be it text or binary
            current = 0;
            mybytearray  = new byte[filesize];
            bytesRead = in.read(mybytearray,0,mybytearray.length);
            current = bytesRead;
            do {
                bytesRead = in.read(mybytearray, current,
                       (mybytearray.length-current));
                if(bytesRead >= 0) current += bytesRead;
            } while(bytesRead > -1);

            // Store recieved data to file, if set true from options
            if (tcp_save_downloaded_file == true) {
                // decide an appropriate file name acc. to download link
                String url = "file:///E:/Data/" + "tcp_downloaded_file.pdf";
                FileConnection fconn = (FileConnection) 
                        Connector.open(url, Connector.READ_WRITE);
                if (!fconn.exists()) {      // XXX. what if file already present? overwrite or append mode?
                    fconn.create();
                }
                OutputStream ops = fconn.openOutputStream();
                ops.write(mybytearray, 0 , current);
                ops.flush();
                ops.close();
            }

        } catch (Exception ex) {
            parent_class.main_form.append("Exception occured while "
                    + "downloading file: " + ex.toString() + "\n\n");
        } finally {
            if (in != null) {
                try {
                    in.close();
                } catch (IOException ex) {
                    parent_class.main_form.append("Exception occured while "
                            + "closing inputstream "
                            + "after downloading file: " + ex.toString() + "\n\n");
                }
            }
            // XXX. see if you need to close the OutputStreams and 
            // SocketConnection as well.
            return server_reply;
        }
    }

Answer 2

The first 10 lines are the HTTP message headers. For more information on them please go to https://www.rfc-editor.org/rfc/rfc2616#page-31.

The blank line identifies where the body starts.

You can start saving the pdf content from line 12 onwards, but you should do it using a different read method. Instead of

data = (byte) in.read();
while (data != -1) {
    ops.write(data);
    data = (byte) in.read();
}

please try

byte buff[] = new byte[1024];
int len = in.read(buff);
while (len > 0) {
    ops.write(buff, 0, len);
    len = in.read(buff);
}