CODEWITHSUNDEEP
javaeclipseurlconnection
Mich
Mich

Reputation: 300

read a huge amount of data

In a java program (Eclipse ISE on a PC) I want to read a huge amount of data (around 1640188 bytes) from a web site. With Wireshark I can see that these datas come in many blocks of 1460 bytes. When I use the following code I read only the first block seen at high level (size around 18000 bytes). How could I do to have the other blocks?

URLConnection con = url.openConnection();
InputStream input = con.getInputStream();
while(input.available()>0)
{
        System.out.println(input.available());
        int n = input.available();
        byte[] mydataTab = new byte[n];
        input.read(mydataTab, 0, n);
        String str = new String(mydataTab);
        memoData += str;
}

Upvotes: 1

Views: 289

Answers (1)

Moises Cavcalcanti
Moises Cavcalcanti

Reputation: 67

First: Do not

int n = input.available(); byte[] mydataTab = new byte[n];

because:

Note that while some implementations of InputStream will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.

Java InputStream Documentation

Second: Try to use some predefined chunck size for your reading, so you can do:

int chuncksize = 1024;
int sizeRead = input.read(mydataTab, 0, n);

where the sizeRead is the amount of bytes that you read. And keep reading the chunks until the end of the streaming.

Upvotes: 3

Related Questions